Building a premium chat experience on Android 8

Think about your favorite chat app experience.
You can send and receive messages, react, reply, see contacts details, browse received and sent media, pin conversations, delete or archive them and much much more. Additionally you can do all this while offline.
All of these features on their own at first don’t sound that complex, but combining all of them, adding the engineering overhead of an offline-first approach, and also supporting devices that are 10+ years old becomes a challenge. Making things worse (for you – the developer), users expect those features to work effortlessly on their device because chat experiences have evolved so much over the years.

In this article, we’ll walk through the roadblocks we hit and how we evolved from relying on SDK storage to a hybrid architecture that scales across devices to achieve a polished chat experience in all performance buckets.

What “Offline-First” Actually Means

Before diving in, it’s worth clarifying what offline-first implies in practice:

• Users can read previously loaded data without a connection
• Users can perform actions while offline
• The system synchronizes changes once connectivity is restored
• The UI remains responsive and consistent, regardless of network state

> We were tasked with building a chat experience so all examples will use chat as an example. Having said that, solutions presented can be applied to other domains if you so desire.

1. Chat

Chat is one of the most demanding offline-first scenarios.

Users expect to:

• Read message history instantly
• Send messages even when offline
• See updates reflected in real time
• Experience zero lag, even with many large conversations

As these things usually go, we didn’t find the best solution to meet all of the above expectations immediately so let’s go on an adventure – together.

An implementation detail

To build our chat feature we’ve decided to use XMTP. You can read more about XMTP here. If you are curious about some gotchas and intricacies I suggest you read an article about things not in the docs written by my colleague Juraj.

Let’s get started!

2. Initial setup

XMTP already has a built-in database (SQLite) which handles data persistence and exposes reactive streams for new data.

This was a good starting point for our use case. Reactive streams gave us flexibility as well as new conversations and messages out of the box.

You can see the initial architecture we’ve used in the Diagram 1.

3. Why didn’t the SDK database scale?

As we’ve mentioned XMTP already had a built-in database which handles data persistence across both platforms and has support for offline sending and syncing.

This sounds like it solves all of our pain points, but unfortunately it’s not so simple.

The SQLite database is great for data persistence, however, we’ve found (the hard way) that access to that database gets painfully slow once the number of chats and incoming messages increase significantly.

In addition, we’ve had issues with updating the state of the UI once receiving new events. Both issues are due to implementation details of the SDK and how it persists data in the database.

So what do you do in this situation? Because you know that CPU cache and RAM are faster than the disk, you introduce an in-memory cache.

We’ve done in-memory caching as SharedFlows. They enable us to replay the same computed value to all subscribers which reduces computation significantly with the added benefit of clearing the cache once subscribers disappear.

We can also share them on a background dispatcher to prevent hogging the main thread. And since they are Flows we can easily combine or merge them to produce values which we need to render the UI.

This was great – the app was noticeably faster, less laggy and janky and conversation changes got applied instantly. We thought we were at the finish line.

Then we ran the app on a lower end device – e.g. an Android 8 (Oreo) device from 2016. For comparison, the Samsung Galaxy S25 Ultra (which was the current flagship Samsung device) has a Geekbench single core performance of 2849, and our low-end device a score of 372. This difference is only accentuated by the ~5x faster storage on our flagship device (UFS 2.0 vs 4.0) which additionally has 3x more RAM than the low end device.

When dealing with more than one page (~15) conversations the low-end device starts to lag, ANRs are popping up everywhere, the phone gets hotter than a toaster and the garbage collector is holding on to dear life. The amount of processing and objects which were kept in memory was too much for that device to handle. We needed to do better.

4. Can pagination fix memory issues?

We didn’t immediately give up on our in-memory solution. After all, it was working well on flagship devices. We’ve rather tried to optimize it.

Enter – pagination.

XMTP SDK supports pagination out of the box. You can query by timestamp and limit the number of entries to whatever you want. To improve the initial load time and reduce memory strain this seemed like the way to go.

What we did:

1. When the page opens load only top 15 recent conversations (to fill out a single page ASAP)
2. As the user scrolls down, load subsequent pages until the end is reached.
3. ???
4. Profit

This solution was good enough. For a while. When testing we’ve found that when scrolling for a long(er) time the device would still lag because the memory had filled up again. Additionally, the algorithm for pagination was quite complex because we’ve had to take into account that conversations which receive messages have timestamps newer than the first item in our list when we started paginating. On top of that the list needs to change the order by the latest message sent in real time which again messes up the order in the list.

Long story short – a hell to maintain and debug with the added bonus of poor performance in certain cases.

Key takeaways

1. In-memory caching improves speed but is limited by hardware
2. Object weight matters
3. Always test your solution in a low-end performance bucket
4. Don’t skip thinking about complexity and maintainability when designing a solution

5. Another database?

Digging deeper we discovered the real issue – the Conversation object was just too heavy to keep in memory. Even with solutions which would release those objects from memory as soon as possible the nature of garbage collection with a very limited memory resulted in the GC causing significant lag. At this point we became aware that we need to introduce a layer between our app and the SDK.

We decided to use an additional database (with Room) to store the bare essentials. This included fields like id, last_message, unread_count, last_updated_millis etc. which were already mapped and optimized for usage in the upper layers of the architecture.

Then, as we receive an update from the SDK we update our database (if needed) which after going through the app layers updates our UI.

Why is this any better?

1. It reduced the Conversation object to a few primitive type fields.
2. Instead of computing new Conversation objects for each update we could update specific columns instead which might (or might not) result in a creation of a new, lightweight, data class.
3. Combining datasets was significantly faster using an efficient SQL query than working with collections (in-memory >1s, SQL ~100-300 ms).
4. When working in-memory we’ve resorted to Maps instead of Lists for Conversations and Members which reduced the time complexity significantly (learn your DSA).
5. We’ve leveraged operators such as .distinctUntilChanged() and mapLatest() to avoid unnecessary UI updates and map computations. When working with hundreds of conversations and thousands of distinct users the performance benefit is obvious.

As you might already know, nothing in life comes for free. To reap all these benefits with this solution we had to solve the problem of stale data. We’ve had to keep our database in sync with the single source of truth which is the SDKs SQLite database. This was no easy task and intricacies of such an implementation is a topic for another article. It is worth mentioning however, that due to a unidirectional data flow of this solution it was much easier to reason about than pagination.

To give you a glimpse into the final solution here is a high level diagram of how the system ended up looking.

Key takeaways

Architecture

SDK – handles its internal logic around sending, syncing and encryption

Data layer – ensures state sync with the SDK and propagating changes to other actors about entity updates

Domain layer – encapsulates the chat feature, defines domain models which are SDK agnostic

Each layer has its own responsibility and functionality which does not trigger if relevant data has not changed thus avoiding unnecessary computation when moving towards the upper layers of the app.

Data structures

When working with large datasets knowing the time complexity of commonly accessed resources matters (e.g. lookup on a List vs Map).

Database

Knowing how to write efficient SQL matters.

SQL queries are faster than mapping in memory.

6. Conclusion

Building premium, offline-first features is less about choosing the “right” tool and more about understanding the constraints of real-world devices. What works perfectly on a flagship phone can completely fall apart on older hardware. Our initial assumption – that the SDK’s built-in database would be enough – quickly broke down under scale. Moving everything into memory improved responsiveness but introduced a different class of problems: memory pressure, garbage collection pauses, and poor performance on low-end devices. Pagination helped, but added complexity without fully solving the issue.

The real breakthrough came when we stopped thinking in terms of a single data source and introduced a layered approach.

By combining the SDK as the source of truth with a lightweight local database tailored to our UI needs, we were able to significantly reduce memory usage, improve performance, and maintain a responsive user experience across devices.

This approach does come with trade-offs. Most notably the need to handle synchronization and stale data, but it provides a much more scalable foundation which is easier to maintain and to reason about.

If there’s one key takeaway, it’s this: premium, offline-first, computation heavy features require balancing between memory, persistence, and synchronization.

There is no one-size-fits-all solution, but with the right abstractions and a willingness to iterate, you can build systems that feel fast, reliable, and resilient for all users, even on older hardware and OS.