package:cronet, an HTTP Dart/Flutter package with dart FFI

My Dart Google Summer of Code 2021 experience.

Dart, Google Summer of Code & package:cronet

package:cronet is an HTTP package for Dart CLI and Flutter platforms. It is powered by Chromium’s network stack, which is written in C/C++. package:cronet born during Google Summer of Code, 2021 with the help of Daco Harkes, Prerak Mann and their student (that is me 😄).

It wasn’t possible without Daniel Stahr from the Chromium team and all the efforts of Chromium authors. 💙

cronet | Flutter Package

This blog post will focus on the technical details of my GSoC project — package:cronet. Working with Googlers and an awesome community of Dart/Flutter devs in this “bleeding-edge” technology is a thrill in itself. There are a lot of things about GSoC apart from these technical stuff (but, let’s write that fragment of the story in a different blog 😅).

So without further delay, let’s get started with the journey of a Google Summer of Code project.

GitHub - google/cronet.dart: This package binds to Cronet's native API to expose them in Dart.

Why package:cronet?

You might be wondering why we need another HTTP package when the standard dart:io already provides us with nice and fast networking. But, package:cronet has the following advantages:

• Faster than dart:io itself in most of the scenarios. Especially when a lot of networking is being done.
• It has QUIC/HTTP3 support. This is one of the highly requested community features that is absent in dart:io.
• Request prioritisation, different kinds of caching, late binding and re-usability of sockets using socket pools. Read more.
• Reliable network operation in unreliable network scenarios. Like, fluctuating speed, switching between mobile data to wifi and vice versa.
• Better support for compressed data formats.
• The core network engine is totally based upon C/C++.
• It is the same networking engine that Chromium uses. (Hence the name Cronet. Chromium Network Stack).

Challenges with callbacks

Cronet is multi-threaded in nature. Whenever a networking event occurs, such as - getting a redirect request, getting some response from the server - it fires a callback along with the event-related data it got. Now, we need to get that data into our Dart code. So, the callback from Cronet must execute Dart code.

However, Dart FFI doesn’t support asynchronous functions yet. Dart FFI calls are synchronous in nature.

Dart code execution is single threaded. Fortunately, Dart has a concept called isolates. Dart code is executed in an isolate, and multiple isolates can all have their own Dart code execution running. It’s like C/C++ threads but with independent (isolated) memory space. Isolates can communicate among each other by sending messages via SendPort and ReceivePort pair.

So I thought, why not just spawn an isolate and execute Cronet callbacks there. But, I was terribly wrong.

Crash in case of C/C++ code invoking Dart code directly

Every isolate ensures that there is only one mutator (i.e. Dart objects can’t be changed from more than 1 thread at the same time) and that implies there should be only one thread executing Dart code. And, just like before, if our C/C++ thread also invokes Dart code, we’re not following the above constraints and hence — crash.

ReceivePorts and event loop to the rescue

After many failed attempts, I stumbled upon ReceivePort. A ReceivePort will have a SendPort and using this SendPort, one can send data to the paired ReceivePort. The data sent via SendPort can be listened to by subscribing to the Stream generated from the ReceivePort.

Streams are handled by one event loop per isolate in Dart. Dart exposes a C API to send data via a NativePort, which is a native representation of SendPort.

Our C code will be invoked by Cronet when any event will occur and in those C callback functions, we will put the received data into the NativePort (i.e. SendPort) using the Dart provided C API. Then, an event will be scheduled to consume it and it will be delivered asynchronously, thanks to the event loop. We will listen to the Stream from ReceivePort and fire a callback on the Dart side accordingly.

However, there was another problem. To send data using Dart native api, we need to modify our shared library, Cronet. But, we can’t modify Cronet. So, the solution is to implement a middleware (or, wrapper), that will pass it’s function address to Cronet as a callback and in that function we will send data to the Dart world.

package:cronet overview

We merged 14 pull requests in this GSoC period and translated this plan into a reality.

Implementing the core features

Let’s now jump into what features we landed during this GSoC 2021 period and how that stacks up against dart:io.

The essentials

In the first PR, we created the base of our package which I primarily built before the coding period as a proof of concept of our plan. This PR was gigantic. In this PR, I implemented the middleware required for the C — Dart asynchronous communication, laid out the whole structure of our package, implemented the most essential APIs, documented them and written tests for them.

After this PR, our minimal viable package was ready to be published on pub.dev and was compatible with 64 bit Desktop environments in Dart CLI.

This PR added support for the following features:

1. HttpClient with QUIC, HTTP2, brotli support.
2. HttpClient with a customizable user agent string.
3. HttpClient close method.
4. open, openUrl & other associated methods.
5. Response using dart:io style API.
6. Different types of Exceptions.

Reference: google/cronet.dart#2, google/cronet.dart#7, google/cronet.dart#22

User-defined headers support

The essentials didn’t support custom HTTP headers and file upload yet. So, we decided to add those. So we planned to solve the later one as the data upload feature will require us to set Content-Type header anyways.

Reference: google/cronet.dart#26

Data Upload

This was the final feature we added during the GSoC period. With this we will be able to send arbitrary data to the server. While implementing this feature, I faced the same set of problems I encountered while implementing the basic GET request. And, the same solution worked (that I’ve described at the very beginning of this post). Apart from this, we also had to copy data from Dart buffer to C buffer. As memcopy is not available yet, we settled for Pointer<Uint8> ‘s extended operator, []= to do our job.

Reference: google/cronet.dart#28

Feature comparison with dart:io

Most, if not all, apps/packages in the Dart ecosystem use dart:io for their networking needs as it is the only library that is baked into Dart SDK itself. package:cronet could potentially be a replacement package of the networking functionality provided by dart:io. That’s why it is important to know where both of them stand compared to each other.

Comparison of common features between package:cronet and dart:io

There are many other features in cronet (i.e. request prioritisation, in-memory caching, etc.) that we are yet to implement and not in the above list.

Compatibility 🤝

The majority of the package:cronet and dart:io APIs is identical, and this is by design. This maximizes compatibility and eases migration. Though there are almost negligible amount of breaking changes, we still have some.

I. SecurityContext

Unlike dart:io, in package:cronet's HttpClient constructor does not take a SecurityContext as a parameter. To set custom SSL certificates, you need to add them in the system’s trust store in that platform’s specific way.

This is due to the fact that cronet only allows the HTTPS connection if the presented SSL certificate is in the system’s trust store and provides no API to add SSL certificates per cronet engine instance.

II. userAgent

package:cronet takes userAgent string via constructor of HttpClient and it is considered final after that. While dart:io exposes it as a property of HttpClient.

This change was required because, we can not mutate the userAgent string after cronet engine has started. userAgent is a part of cronet engineParameters that is required to initiate cronet itself.

The breaking changes can be tracked at cronet.dart/dart_io_comparison.md.

Making it cross-platform

Initially we started with supporting Dart CLI on 64 bit Desktop platforms. We also added support for other Dart Native platforms.

Adding Flutter support

This was one of the most community desired features. In this PR, we added support for Android, Windows, Linux and Mac support in Flutter. iOS support is only 1PR away.

Here also we faced a few challenges. In Android, directly loading the Cronet shared library from Dart didn’t work. We need to add Cronet as a dependency in Gradle & load cronet’s shared library from both Kotlin and Dart though we’re not using JNI.

For desktop environments, we didn’t have to do anything special. Though when we added flutter keys into the pubspec.yaml file for having android build, we also needed to explicitly mention the flutter version. With this change, we lost the chance of doing dart pub. We only have flutter pub for now even if we do want to use it in a Dart CLI project. Though, this issue will be fixed later as we make progress in dart-lang/pub#2606.

Reference: google/cronet.dart#18

Adding 32 bit architecture support

Initially, we only focused on 64 bit systems as they are most widely used nowadays and Flutter does not have a 32 bit SDK (for desktops) anyways. However, we felt adding it for supporting 32 bit Dart SDK and most importantly — Android Emulator (which is x86 based) which is used in Flutter, so if you want to test without a device you need this.

Initially, we were making an array of pointers that held the pointers to the callback arguments. The array was a type of uint64_t that we are consuming on the Dart side. In this PR, we interpreted the pointers as uintptr_t. In the 32 bit execution environments, we ensured that the pointer addresses get stored in the lower 32 bit of a uint64_t storage space. Then we can continue using Uint64List on the Dart side for both the architectures.

Reference: google/cronet.dart#24

Benchmarking

To know how package:cronet really holds up against dart:io, we needed few benchmarks. Cronet is not integrated into the Dart VM which might create some overhead. On the other hand Cronet provides HTTP 2 and QUIC. So, we came up with two different flavours of benchmarking.

We will also take both Just in Time (JIT) & Ahead of Time (AOT) compilation methods. When we do Flutter development (using flutter run) or run a Dart CLI program (using dart run), the code is compiled in JIT mode. However, a store distributed Flutter app or dart compile exe produced binary (which is used in production) contains AOT compiled code. So, both compilation modes are important to us.

Latency benchmark

In this type of benchmark, we were interested in measuring how long a single request takes. It is a pretty common scenario in an application’s lifecycle to do multiple network calls in a sequential manner.

Throughput benchmark

In the throughput benchmark, we tried to observe how both of them perform in terms of parallel network requests. In network heavy apps, a general use case is having multiple network dependent components in the same screen. And, efficiently doing parallel requests are important to make those apps snappy.

Comparison with dart:io

We performed the benchmark in different scenarios based on — network speed, cpu and memory capacity and OSes. Sometimes these results varied and can be seen in details at google/cronet.dart#3.

Here we present the result from a general purpose machine (Ryzen 5 2500U, 8GB RAM) with a 1.7MBPS network connection running Ubuntu 20.04 against example.com.

Performance comparison between dart:io and package:cronet

Throughput benchmark was capped at 128 parallel requests for both package:cronet and dart:io.

Comparing app size

We built the flutter example app and compared the release apk build size for both package:cronet and dart:io. It turns out, using package:cronet bumps the app’s size from 5.5MB to 7.5MB. The increment of 2MB and mostly due to the cronet shared library we’ve to ship along with the apk and this size bump is mostly constant.

There can be few potential angles to reduce app size (in future).

1. Removing networking components of dart:io if we are using package:cronet.
2. If we can compile a subset of Cronet on need basis.
3. If we could somehow load the Cronet already available on Android devices we would have no size increase at all.

However, these were outside the scope of the GSoC project.

Dart best practices

Following best practices are always important to develop a maintainable and less-prone-to-bugs code. And also, saves a lot of future work by not re-designing what is already designed.

Code style consistency using package:lints

Initially, we started with pedantic for linting rules. Later we moved to package:lints to follow a stricter subset of Effective Dart guide.

Reference: google/cronet.dart#2

Using package:args

With essential cli utilities, comes great argument parsing responsibilities. package:cronet has a lot of helper utilities that use Dart CLI very often. From setup phase to easy benchmarking — cli is our friend in need.

Though we started with my implementation of parsing, later we moved to package:args. This package from the Dart team made the CLI interface more consistent, reduced the number of lines of code and made it easier to manage.

Reference: google/cronet.dart#15

Learning

Though I’ve talked a lot about the project and learning already, there are few more I can add. These learning may not be specific to this project, but nonetheless, valuable. package:cronet is my life’s first package (in any language) and my first time working with ffi. So, some of the learning may seem trivial.

• How an HTTP library performs its task and learnt about its lifecycle.
• How can we properly document a codebase.
• How to handle function pointers in C/C++.
• How data is represented differently in C/C++ — Dart and how we can perform conversion between them.
• What are the finalisers and how Dart garbage collector works.
• What are the best practices to follow when developing a Dart package.
• How a software is made and managed in real life.
• How can we ensure cross compatibility.
• How can we debug when something isn’t working and how to deal with edge cases.

Apart from these, I also learnt about multithreading, internals of streams and futures, designing APIs, etc. I’ve improved on my communication skills too.

Future of package:cronet

10 weeks aren’t the end of the package:cronet. We have few (non exhaustive) future plans that we want to talk about.

• Having Async callback support in Dart FFI will help us remove the wrapper code.
• Reducing the application size.
• Addressing the issues with package distribution (e.g. providing binaries and having both dart run and flutter run capabilities in native plugins).
• Adding network request prioritization feature.
• Covering all of dart:io APIs and introducing most requested community features (if feasible).
• Discovering how we can migrate dart:io based packages (e.g. package:http) to cronet based ones.

Mentorship

Daco Harkes — He is a Software Engineer from Google, currently working on Dart VM. He is the one who made me understand Dart from the core. Despite the timezone and language differences we have, he is very responsive in helping me out whenever I’m stuck. He walked me through various challenges, describing when something works (and why) and when it doesn’t. The level of trust and understandability he shows when I mess something up is astounding.

Prerak Mann — He is a Software Development Engineer at Pickrr and was a GSoC 2020 student under Daco’s mentorship. He always helped me whenever I’m stuck, suggesting ideas and portions that I overlooked. He and his GSoC project, package:ffigen is a lifesaver for package:cronet. From adding MacOS support to the package:cronet and the pointers he gave, made by GSoC experience so much smoother.

I also want to thank Daniel Stahr (Senior Software Engineer at Google) for the help he provided despite not being my GSoC mentor (officially). It wasn’t possible for me to demystify cronet without him. He literally resolved my doubts in minutes whenever I needed.

Resources

There are few resources I want to introduce that helped me in this project.

• Dart asynchronous programming: Isolates and event loops
• Isolates and Event Loops — Flutter in Focus
• Build and deploy native C libraries with Flutter
• Cronet request lifecycle
• Cronet native sample — Chromium source code
• C interop using dart:ffi
• C interoperability with Dart FFI | Session

Also, Daniel’s examples regarding cronet’s APIs and Sunbreak’s work helped us a lot.

Lastly, I would like to thank Daco, Prerak, Daniel, Jonas, the Dart team, the GSoC team and the whole community at large for making this summer awesome 😎.