How to Build AI Agents with the Flutter AI Toolkit

We build Flutter apps for a living at Dartitude, so treat this as a practitioner's walkthrough with a clear bias: we like keeping the whole feature in Dart instead of standing up a separate service just to talk to a model. If that bias doesn't fit your stack, the architecture section below tells you where the seams are.

What the Flutter AI Toolkit actually is

The Flutter AI Toolkit is an open-source package from the Flutter team. The current release is flutter_ai_toolkit 1.0.0, licensed BSD-3-Clause, and it lives in the flutter/ai repository. The official description is plain: it's a set of AI chat widgets that make it easy to add an AI chat window to a Flutter app, targeting mobile, desktop, and web.

Here's the part people miss. The toolkit is a chat UI plus a provider abstraction. It is not, by itself, an agent framework. The official docs list what it gives you out of the box:

• Multiturn chat that keeps context across messages
• Streaming responses that render as tokens arrive
• Rich text display for formatted replies
• Voice input
• Multimedia attachments
• Function calling, meaning the model can request one of your tools

That last item is the one that matters for agents. Everything else is a very good chat window.

Chatbot or agent? The difference in one line

A chatbot answers. An agent acts.

Ask a chatbot to cancel a subscription and it explains the steps. Ask an agent and it calls your cancelSubscription function, reads the result, and confirms. The gap between those two is function calling plus a little orchestration. Four pieces separate a real agent from a pretty chat box:

1. Tools the model can invoke (your app's real actions)
2. Memory so the agent remembers earlier turns
3. A loop that runs tools, feeds results back, and continues until the model is done
4. Guardrails that stop it doing something dumb or expensive

The good news for Flutter developers is that as of version 1.0.0, the toolkit handles the loop for you. The changelog puts it bluntly: the FirebaseProvider now has the inner loop to keep calling the functions the model requests until it has enough to answer. The Flutter team called that "the beating heart of an AI agent," and they're right. You used to write that loop by hand. Now you define tools and let the provider drive.

Architecture: know where the seams are

Think in three layers. This keeps your code testable and stops the agent logic from leaking into your widgets.

The toolkit sits in the top and bottom layers. The middle layer is yours, and it's where your product actually gets smart. If you find agent logic creeping into a widget's build method or a button callback, pull it back out into a service. That one habit prevents most of the maintenance pain teams hit later.

Setup, step by step

You need a Firebase project because the toolkit's built-in provider routes through Firebase AI Logic. Both the prototyping endpoint (Gemini Developer API) and the production endpoint (Vertex AI) require it, along with firebase_core.

Step 1. Add the dependencies.

dependencies:
  flutter_ai_toolkit: ^1.0.0
  firebase_ai: ^latest_version
  firebase_core: ^latest_version

Run flutter pub get, then connect Firebase with the flutterfire CLI as the Firebase AI Logic docs describe.

Step 2. Initialize Firebase before runApp.

import 'package:firebase_core/firebase_core.dart';
import 'package:flutter_ai_toolkit/flutter_ai_toolkit.dart';
import 'firebase_options.dart'; // generated by flutterfire config

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  await Firebase.initializeApp(
    options: DefaultFirebaseOptions.currentPlatform,
  );
  runApp(const App());
}

Step 3. Drop in a working chat.

import 'package:firebase_ai/firebase_ai.dart';
import 'package:flutter_ai_toolkit/flutter_ai_toolkit.dart';

class ChatPage extends StatelessWidget {
  const ChatPage({super.key});

  @override
  Widget build(BuildContext context) => Scaffold(
        appBar: AppBar(title: const Text('Assistant')),
        body: LlmChatView(
          provider: FirebaseProvider(
            model: FirebaseAI.googleAI().generativeModel(
              model: 'gemini-2.5-flash',
            ),
          ),
        ),
      );
}

That's a full streaming chatbot in about a dozen lines. It talks, it remembers the conversation, it renders markdown. It just can't do anything yet. Let's fix that.

Function calling: turning the chat into an agent

An agent needs actions. In the toolkit, an action is a FunctionDeclaration you hand to the model, plus an onFunctionCall callback that runs the real Dart code when the model asks for it.

Here's a small agent with two tools. This is the current, verified pattern from the toolkit's own function calling example.

LlmChatView(
  provider: FirebaseProvider(
    model: FirebaseAI.googleAI().generativeModel(
      model: 'gemini-2.5-flash',
      tools: [
        Tool.functionDeclarations([
          FunctionDeclaration(
            'get_temperature',
            'Get the current local temperature',
            parameters: {},
          ),
          FunctionDeclaration(
            'get_time',
            'Get the current local time',
            parameters: {},
          ),
        ]),
      ],
    ),
    onFunctionCall: _onFunctionCall,
  ),
)

Now the handler. When the model decides it needs a tool, the provider calls back into this function, and whatever you return goes back to the model.

Future<Map<String, Object?>> _onFunctionCall(
  FunctionCall call,
) async {
  switch (call.name) {
    case 'get_temperature':
      final celsius = await weatherService.currentTemp();
      return {'temperature_c': celsius};
    case 'get_time':
      return {'time': DateTime.now().toIso8601String()};
    default:
      return {'error': 'Unknown tool: ${call.name}'};
  }
}

That's the whole agent. The user asks "is it cold out?", the model calls get_temperature, your handler returns a number, and the model turns that number into a sentence. If it needs the time too, it asks for that in the same turn. The FirebaseProvider keeps looping through tool requests until the model stops asking and writes its final reply.

Notice what you did not write: no manual detection of function calls, no hand-rolled loop, no re-sending history. The toolkit owns that now. Your job is the two things only you can know, which tools exist and what they do.

Memory: the thing that fails silently

The toolkit keeps conversation history for you, which is great until a long chat quietly hits the model's context limit and older messages fall off without a warning. We've watched this bite teams in production more than once.

The fix is boring and it works: cap how much history you send. Keep a rolling window of recent turns, or summarize older ones into a short note and prepend that. You don't need a library. A function that trims the message list before each request is enough for most apps. Add it before your first real user, not after.

How the toolkit compares to the alternatives

You have three honest options for adding an agent to a Flutter app. Here's how they stack up.

A quick caution on names. There are community packages with similar titles, such as flutter_ai_agent_tool and a project simply called flutter_ai. They are not the official toolkit and have different APIs. If a tutorial imports something other than package:flutter_ai_toolkit, it's not describing this package. That confusion trips up a lot of first-time searchers.

Real use cases we'd actually build

These aren't hypotheticals. Each maps to a tool set you could ship this quarter.

• In-app support agent. Tools like getAccountStatus, applyCredit, and createTicket. The agent resolves the routine billing question and escalates the rare hard one. Fewer tickets reach a human.
• Field data-entry agent. A technician finishes a job and speaks a summary. Tools transcribe it, pull out the structured fields, and pre-fill the form. The person confirms instead of typing on a phone in the rain.
• Developer helper. A tool that fetches a diff and another that runs a linter. The agent reads both and returns readable feedback, rendered as markdown in the same chat view.

The pattern is the same every time. List the actions, write them as functions, declare them, ship.

Common mistakes to avoid

We've made most of these ourselves, so learn from them cheaply.

• Putting agent logic in widgets. Keep tools and orchestration in a service, not in onSendMessage. Your future self will thank you.
• Ignoring token limits. History grows every turn. Trim it from day one.
• Committing secrets. Don't check firebase_options.dart into a public repo when the app calls Gemini straight from the client. The Flutter docs flag this directly.
• Calling the model from the client in production. For real workloads, route requests through a backend such as Cloud Functions or Cloud Run. The client-only setup is for prototyping.
• No error path in tools. If a tool throws and you don't catch it, the loop can stall. Return a structured error so the model can recover or apologize.

Should you use it?

If you're building a chat-shaped feature and you're comfortable in the Google and Firebase ecosystem, the toolkit is the fastest correct path. You get streaming, history, attachments, voice, and a real tool-calling loop for the cost of a widget and a provider. You spend your time on the tools that make your product yours, which is exactly where the time should go. If you need a non-chat interface, multiple model vendors, or a UI that doesn't look like the default, you'll be happier with the raw SDK or a third-party package. Neither is wrong. Pick the one that matches the shape of your feature.

Building something with real agent requirements and short on Flutter hands? That's what we do. Here's how teams hire a Flutter app development company that has shipped this kind of work.

Frequently asked questions

1. Is the Flutter AI Toolkit free and open source?

Yes. It's published by the Flutter team under the BSD-3-Clause license and is available on pub.dev as flutter_ai_toolkit. You can read, fork, and contribute to the source in the flutter/ai repository.

2. Does it support OpenAI or Claude, or only Gemini?

Out of the box, the built-in FirebaseProvider connects to Google's Gemini and Vertex AI through Firebase AI Logic. Other vendors like OpenAI or Anthropic aren't supported by default. You can add them by implementing the LlmProvider interface yourself, or use a third-party package built for multiple providers.

3. Do I need a backend to use it?

Not for prototyping. The toolkit can call Gemini directly from the client. For production, the Flutter docs recommend routing requests through a backend such as Cloud Functions for Firebase or Cloud Run, mainly to protect keys and control usage.

4. How is this different from just calling the Gemini API myself?

The toolkit gives you the chat UI, streaming, history management, attachments, voice input, and the tool-calling loop as ready-made pieces. Calling the API yourself means building all of that. If your feature is chat-shaped, the toolkit saves weeks.

5. Can agents run offline?

No. The agent loop depends on live calls to the model, so it needs connectivity. Design a clear fallback state for when the device is offline rather than letting requests hang.

6. Does function calling work on both iOS and Android?

Yes. The toolkit targets mobile, desktop, and web from one codebase, and function calling works the same across them because the logic runs in Dart and the model calls go through Firebase.