← Back to posts Cover image for Building Robust Flutter Apps: Strategies for Handling Diverse External Data Sources

Building Robust Flutter Apps: Strategies for Handling Diverse External Data Sources

· 12 min read
Chris
By Chris

Flutter apps often don’t live in a vacuum. They need to talk to the outside world, pulling in data from various places – APIs, local files, even user-uploaded documents. This is where things can get tricky, especially when those external sources come in a multitude of formats and structures.

Imagine you’re building a sales analytics app, a common scenario for developers looking to track business performance. Your users want to upload their sales data from different e-commerce platforms. Shopify provides a CSV with columns like Date, Product, Quantity, Price, CustomerName. Etsy offers an Excel file with OrderDate, Item, Units, SalePrice, Buyer. WooCommerce has an API returning JSON with order_id, product_name, item_count, total_price, customer_email.

The core problem often arises: “Do I need separate import logic for every platform?” If you’re not careful, you might end up with a tangled mess of conditional logic, making your app a nightmare to maintain and extend.

The answer to that question is both “yes” and “no.” Yes, you will need specific logic to understand each unique format. But no, this doesn’t mean your entire application needs to be aware of every single platform’s quirks. The trick lies in abstraction and decoupling.

The Core Principle: A Unified Domain and Adaptable Data Sources

The golden rule here is to define a common domain interface that represents the data your app cares about. This is your app’s internal language for sales, customers, or whatever entities you’re managing. Then, for each external source, you create an adapter (or parser) that translates that source’s specific format into your app’s common language.

Let’s break this down with our sales analytics example.

Step 1: Define Your App’s Domain Model

First, forget about platforms, CSVs, or JSON for a moment. What does a “sale” look like inside your app? This is your domain entity.

// lib/domain/entities/sale.dart
class Sale {
  final String id; // A unique ID for the sale, perhaps generated by the app
  final DateTime orderDate;
  final String productName;
  final int quantity;
  final double unitPrice; // Price per item
  final double totalPrice; // Total for this specific line item (quantity * unitPrice)
  final String? customerName; // Optional
  final String? customerEmail; // Optional

  Sale({
    required this.id,
    required this.orderDate,
    required this.productName,
    required this.quantity,
    required this.unitPrice,
    required this.totalPrice,
    this.customerName,
    this.customerEmail,
  });

  // Example factory for creating a Sale from a map, useful for internal data
  factory Sale.fromMap(Map<String, dynamic> map) {
    return Sale(
      id: map['id'] as String,
      orderDate: DateTime.parse(map['orderDate'] as String),
      productName: map['productName'] as String,
      quantity: (map['quantity'] as num).toInt(),
      unitPrice: (map['unitPrice'] as num).toDouble(),
      totalPrice: (map['totalPrice'] as num).toDouble(),
      customerName: map['customerName'] as String?,
      customerEmail: map['customerEmail'] as String?,
    );
  }

  Map<String, dynamic> toMap() {
    return {
      'id': id,
      'orderDate': orderDate.toIso8601String(),
      'productName': productName,
      'quantity': quantity,
      'unitPrice': unitPrice,
      'totalPrice': totalPrice,
      'customerName': customerName,
      'customerEmail': customerEmail,
    };
  }
}

This Sale class is now the single source of truth for what a sale means in your application. No matter where the data comes from, it must eventually conform to this structure.

Step 2: Abstract Your Data Sources

Next, define an abstract interface for how your application retrieves sales. This is your data source contract.

// lib/data/datasources/sale_data_source.dart
import '../../domain/entities/sale.dart';

abstract class SaleDataSource {
  Future<List<Sale>> getSales();
  // You might have other methods like getSalesByDateRange, saveSales, etc.
}

// Optional: for file-based sources, you might want a specific interface
abstract class FileSaleDataSource extends SaleDataSource {
  Future<List<Sale>> parseFile(String filePath);
  Future<List<Sale>> parseString(String fileContent); // For web uploads or paste
}

This SaleDataSource interface ensures that any class implementing it will provide a list of Sale objects, regardless of its internal workings.

Step 3: Implement Specific Adapters (Parsers)

Now comes the “separate logic” part, but it’s contained within its own adapter. Let’s create an adapter for a CSV file from “Shopify.” We’ll need the csv package (flutter pub add csv).

// lib/data/datasources/shopify_csv_sale_data_source.dart
import 'package:csv/csv.dart';
import 'package:uuid/uuid.dart'; // For generating unique IDs
import 'package:flutter/services.dart' show rootBundle; // For loading local assets

import '../../domain/entities/sale.dart';
import 'sale_data_source.dart';

class ShopifyCsvSaleDataSource implements FileSaleDataSource {
  final String _csvContent; // Or filePath if reading from disk
  final Uuid _uuid; // To generate unique IDs for each sale

  ShopifyCsvSaleDataSource(this._csvContent) : _uuid = Uuid();

  @override
  Future<List<Sale>> getSales() async {
    return parseString(_csvContent);
  }

  @override
  Future<List<Sale>> parseFile(String filePath) async {
    // In a real app, you'd read from filePath (e.g., using dart:io File)
    // For demonstration, let's assume _csvContent is already set from a file.
    // Or load from assets:
    final String fileContent = await rootBundle.loadString(filePath);
    return parseString(fileContent);
  }

  @override
  Future<List<Sale>> parseString(String fileContent) async {
    final List<List<dynamic>> rowsAsListOfValues =
        const CsvToListConverter().convert(fileContent);

    if (rowsAsListOfValues.isEmpty) {
      return [];
    }

    // Assuming the first row is the header
    final List<String> headers = rowsAsListOfValues[0].map((e) => e.toString()).toList();
    final List<List<dynamic>> dataRows = rowsAsListOfValues.sublist(1);

    final List<Sale> sales = [];
    for (final row in dataRows) {
      try {
        final Map<String, dynamic> rowMap = {};
        for (int i = 0; i < headers.length; i++) {
          rowMap[headers[i].trim()] = row[i];
        }

        final DateTime orderDate = DateTime.parse(rowMap['Date'] as String);
        final String productName = rowMap['Product'] as String;
        final int quantity = (rowMap['Quantity'] as num).toInt();
        final double unitPrice = (rowMap['Price'] as num).toDouble();
        final double totalPrice = quantity * unitPrice; // Calculate total for line item
        final String? customerName = rowMap['CustomerName'] as String?;

        sales.add(
          Sale(
            id: _uuid.v4(), // Generate a unique ID for this imported sale
            orderDate: orderDate,
            productName: productName,
            quantity: quantity,
            unitPrice: unitPrice,
            totalPrice: totalPrice,
            customerName: customerName,
            // customerEmail is not available in this CSV example
          ),
        );
      } catch (e) {
        // Log the error, skip malformed rows, or throw a custom parsing exception
        print('Error parsing row: $row - $e');
        // Depending on requirements, you might want to collect these errors
        // and report them to the user.
      }
    }
    return sales;
  }
}

You would create similar classes for EtsyExcelSaleDataSource (using a package like excel_to_json or excel), and WooCommerceApiSaleDataSource (using http and dart:convert). Each one focuses only on translating its specific input format into your Sale domain model.

Step 4: The Repository Pattern

To further abstract where the data comes from, we introduce the Repository pattern. The repository acts as a mediator between your application’s business logic (use cases) and the actual data sources. It decides which data source to use.

// lib/domain/repositories/sale_repository.dart
import '../entities/sale.dart';

abstract class SaleRepository {
  Future<List<Sale>> getAllSales();
  Future<void> importSalesFromFile(String filePath, String platformType);
  // Add other methods like saveSale, deleteSale, etc.
}

Now, the implementation of the repository:

// lib/data/repositories/sale_repository_impl.dart
import '../../domain/entities/sale.dart';
import '../../domain/repositories/sale_repository.dart';
import '../datasources/sale_data_source.dart';
import '../datasources/shopify_csv_sale_data_source.dart';
// Import other specific data sources here

class SaleRepositoryImpl implements SaleRepository {
  // In a real app, you might have a local storage data source too
  final List<Sale> _internalSales = []; // Simulating an internal cache/database

  // Constructor can take a default data source or manage multiple
  SaleRepositoryImpl();

  @override
  Future<List<Sale>> getAllSales() async {
    // In a real scenario, this would fetch from a database or a primary API
    return _internalSales;
  }

  @override
  Future<void> importSalesFromFile(String filePath, String platformType) async {
    FileSaleDataSource dataSource;

    // This is where you decide which parser to use based on platformType
    // In a more complex system, this might be a factory or a DI container
    switch (platformType.toLowerCase()) {
      case 'shopify_csv':
        // For local files, you'd read content first, then pass to parser
        // For simplicity, let's assume filePath directly points to asset content
        // In a real app, you'd load the file content from disk first.
        final String fileContent = await _readFileContent(filePath);
        dataSource = ShopifyCsvSaleDataSource(fileContent);
        break;
      // case 'etsy_excel':
      //   dataSource = EtsyExcelSaleDataSource(filePath);
      //   break;
      // case 'woocommerce_api':
      //   dataSource = WooCommerceApiSaleDataSource(apiClient); // Needs an API client
      //   break;
      default:
        throw ArgumentError('Unknown platform type: $platformType');
    }

    final List<Sale> importedSales = await dataSource.getSales();
    _internalSales.addAll(importedSales); // Add to our internal list/DB
    print('Imported ${importedSales.length} sales from $platformType');
  }

  // Helper to simulate reading file content from an asset
  // In a real app, you'd use dart:io or file_picker for user-selected files.
  Future<String> _readFileContent(String filePath) async {
    // Example: loading from assets for demonstration
    // If it's a user-uploaded file, you'd get its content directly.
    return 'Date,Product,Quantity,Price,CustomerName\n'
           '2023-01-01,T-Shirt,2,20.00,Alice Smith\n'
           '2023-01-05,Coffee Mug,1,15.00,Bob Johnson';
  }
}

Step 5: Integrating with Your UI (Presentation Layer)

Your UI (e.g., a BLoC, Cubit, or ViewModel) interacts only with the SaleRepository. It doesn’t know or care if the sales came from a CSV, an API, or an Excel file.

// lib/presentation/bloc/sale_import_bloc.dart
import 'package:flutter_bloc/flutter_bloc.dart';
import '../../domain/repositories/sale_repository.dart';

// Events and States for the Bloc...
abstract class SaleImportEvent {}
class ImportSalesStarted extends SaleImportEvent {
  final String filePath;
  final String platformType;
  ImportSalesStarted({required this.filePath, required this.platformType});
}

abstract class SaleImportState {}
class SaleImportInitial extends SaleImportState {}
class SaleImportLoading extends SaleImportState {}
class SaleImportSuccess extends SaleImportState {
  final int importedCount;
  SaleImportSuccess(this.importedCount);
}
class SaleImportFailure extends SaleImportState {
  final String message;
  SaleImportFailure(this.message);
}

class SaleImportBloc extends Bloc<SaleImportEvent, SaleImportState> {
  final SaleRepository saleRepository;

  SaleImportBloc({required this.saleRepository}) : super(SaleImportInitial()) {
    on<ImportSalesStarted>(_onImportSalesStarted);
  }

  Future<void> _onImportSalesStarted(
    ImportSalesStarted event,
    Emitter<SaleImportState> emit,
  ) async {
    emit(SaleImportLoading());
    try {
      await saleRepository.importSalesFromFile(event.filePath, event.platformType);
      // After import, you might want to refresh the list of all sales
      // This would typically involve another bloc or a direct call to the repo.
      emit(SaleImportSuccess(0)); // In a real app, return actual imported count
    } catch (e) {
      emit(SaleImportFailure(e.toString()));
    }
  }
}

In your UI:

// lib/presentation/pages/import_page.dart
import 'package:flutter/material.dart';
import 'package:flutter_bloc/flutter_bloc.dart';
import '../bloc/sale_import_bloc.dart';
import '../../data/repositories/sale_repository_impl.dart'; // For dependency injection

class ImportPage extends StatelessWidget {
  const ImportPage({super.key});

  @override
  Widget build(BuildContext context) {
    // In a real app, use a proper DI solution (e.g., GetIt, Provider)
    final SaleRepositoryImpl saleRepository = SaleRepositoryImpl();

    return BlocProvider(
      create: (context) => SaleImportBloc(saleRepository: saleRepository),
      child: Scaffold(
        appBar: AppBar(title: const Text('Import Sales')),
        body: Center(
          child: Column(
            mainAxisAlignment: MainAxisAlignment.center,
            children: [
              ElevatedButton(
                onPressed: () {
                  // Simulate file picking and platform selection
                  // In a real app, use file_picker package
                  const String dummyFilePath = 'assets/shopify_sales.csv';
                  const String selectedPlatform = 'shopify_csv';
                  context.read<SaleImportBloc>().add(
                    ImportSalesStarted(
                      filePath: dummyFilePath,
                      platformType: selectedPlatform,
                    ),
                  );
                },
                child: const Text('Import Shopify CSV'),
              ),
              // Add buttons for other platforms/formats
              BlocBuilder<SaleImportBloc, SaleImportState>(
                builder: (context, state) {
                  if (state is SaleImportLoading) {
                    return const CircularProgressIndicator();
                  } else if (state is SaleImportSuccess) {
                    return Text('Successfully imported ${state.importedCount} sales!');
                  } else if (state is SaleImportFailure) {
                    return Text('Import failed: ${state.message}', style: const TextStyle(color: Colors.red));
                  }
                  return const SizedBox.shrink();
                },
              ),
            ],
          ),
        ),
      ),
    );
  }
}

Handling Different Formats

  • CSV: As shown, the csv package is your friend. Pay attention to headers, delimiters, and potential variations in column order.
  • Excel (XLSX/XLS): Packages like excel_to_json or excel can read .xlsx files. They often provide row/column access, which you then map to your domain model similar to the CSV example. If you only need simple data, sometimes converting Excel to CSV first (either manually or programmatically) is an easier path.
  • JSON (APIs): Standard http package for network requests and dart:convert for jsonDecode. Your API data source would make HTTP calls and then parse the JSON response into your Sale objects.
// Example for an API data source
// lib/data/datasources/woocommerce_api_sale_data_source.dart
import 'dart:convert';
import 'package:http/http.dart' as http;
import 'package:uuid/uuid.dart';

import '../../domain/entities/sale.dart';
import 'sale_data_source.dart';

class WooCommerceApiSaleDataSource implements SaleDataSource {
  final http.Client client;
  final String baseUrl;
  final Uuid _uuid;

  WooCommerceApiSaleDataSource({required this.client, required this.baseUrl}) : _uuid = Uuid();

  @override
  Future<List<Sale>> getSales() async {
    final response = await client.get(Uri.parse('$baseUrl/orders')); // Assuming an /orders endpoint

    if (response.statusCode == 200) {
      final List<dynamic> ordersJson = jsonDecode(response.body);
      final List<Sale> sales = [];

      for (final orderJson in ordersJson) {
        // WooCommerce API often returns orders with line items
        // We'll flatten this into individual 'Sale' objects for our app's domain
        final String customerEmail = orderJson['customer_email'] as String;
        final String? customerFirstName = orderJson['billing']['first_name'] as String?;
        final String? customerLastName = orderJson['billing']['last_name'] as String?;
        final String? customerName = (customerFirstName != null && customerLastName != null)
            ? '$customerFirstName $customerLastName'
            : customerFirstName ?? customerLastName;

        final DateTime orderDate = DateTime.parse(orderJson['date_created'] as String);

        for (final lineItem in orderJson['line_items']) {
          try {
            final int quantity = (lineItem['quantity'] as num).toInt();
            final double itemTotal = (double.tryParse(lineItem['total'] as String) ?? 0.0);
            final double unitPrice = quantity > 0 ? (itemTotal / quantity) : 0.0;

            sales.add(
              Sale(
                id: _uuid.v4(), // Generate ID as line item ID might not be unique across orders
                orderDate: orderDate,
                productName: lineItem['name'] as String,
                quantity: quantity,
                unitPrice: unitPrice,
                totalPrice: itemTotal,
                customerName: customerName,
                customerEmail: customerEmail,
              ),
            );
          } catch (e) {
            print('Error parsing WooCommerce line item: $lineItem - $e');
            // Continue to next line item or order
          }
        }
      }
      return sales;
    } else {
      throw Exception('Failed to load sales from API: ${response.statusCode}');
    }
  }
}

Error Handling and Robustness

  • Validation: Always validate data after parsing. Is the date format correct? Is the quantity a positive number? If not, skip the record, log the error, and potentially inform the user.
  • Try-Catch: Wrap parsing logic in try-catch blocks to gracefully handle malformed data or unexpected formats.
  • User Feedback: When importing files, provide clear feedback to the user: “Import successful: X sales imported, Y sales skipped due to errors.”
  • Network Errors: For API sources, handle network issues (no internet, server down) with appropriate error messages.

Future-Proofing and Maintainability

This architecture offers significant benefits:

  1. Isolation: Changes to Shopify’s CSV format only affect ShopifyCsvSaleDataSource. The rest of your app remains untouched.
  2. Extensibility: Adding a new platform (e.g., Magento) is straightforward: create MagentoSaleDataSource, implement its parsing logic, and add it to your SaleRepository’s factory/switch statement.
  3. Testability: Each data source adapter can be tested in isolation, mocking the raw input data to ensure it correctly translates to your domain model.
  4. Clarity: Your domain layer and UI remain clean and focused on business logic, not on the intricacies of external data formats.

Common Mistakes to Avoid

  • Tight Coupling: Don’t embed parsing logic directly into your BLoCs, Cubits, or UI widgets. This makes your code brittle and hard to change.
  • Ignoring the Domain Model: Jumping straight into parsing without defining your app’s core data

This blog is produced with the assistance of AI by a human editor. Learn more

Related Posts

Cover image for Building Fluid & Interactive UIs in Flutter: Beyond Basic Animations with Custom Painters and Game-Inspired Techniques

Building Fluid & Interactive UIs in Flutter: Beyond Basic Animations with Custom Painters and Game-Inspired Techniques

This post will guide developers through creating highly dynamic and visually rich user interfaces using advanced Flutter techniques like CustomPainter, TickerProviderStateMixin, and even drawing inspiration from game development libraries like Flame for effects. We'll explore how to achieve smooth, interactive animations and reactive UIs that feel truly "liquid" without necessarily building a game.

Cover image for Mastering Responsive & Adaptive Layouts in Flutter: Beyond `MediaQuery`

Mastering Responsive & Adaptive Layouts in Flutter: Beyond `MediaQuery`

This post will guide developers through building truly adaptive Flutter UIs that seamlessly adjust to different screen sizes, orientations, and platforms. We'll cover advanced techniques using `LayoutBuilder`, `CustomMultiChildLayout`, and `Breakpoints` to create flexible, maintainable layouts, moving beyond basic `MediaQuery` checks.