Project 1: llmDraft: LLM-assist Chatter

Course Schedule

We will build an app called Chatter and its corresponding back-end server, chatterd. Users post text messages, called chatts, to the back-end server, which stores each chatt in a database and shares all posted chatts with everyone accessing the back end (be mindful of what you post!). The back-end server, aside from a database to store chatts, also has access to an LLM. A user can ask the LLM to rewrite a chatt they’ve drafted prior to posting, or a user can ask the LLM to draft a reply to a chatt posted by someone else.

Expected behavior

Displaying posted chatts, requesting rewrite of a draft chatt, requesting draft reply of another user’s chatt:

Objectives

In addition to the objectives listed in the llmPrompt and Chatter tutorials, this project has the following objectives for the front end:

• Practice and apply the objectives learned in the two tutorials
• Learn how to implement the long-press gesture

Features and requirements

To receive full credit, your solution app must provide the following features and satisfy the following requirements, including those in any applicable “Implementation guidelines” documents.

Front-end UI

As can be seen in the video above, the Chatter app consists of a single screen with the following UI elements:

1. a title bar showing the title Chatter,
2. a timeline of posted chatts with the current user’s chatt shown on the right and those of other user(s) shown on the left,
   • each chatt is shown in a message “bubble” with its timestamp at the bottom of the “bubble” in smaller fonts and different color from the message’s color,
   • the chatt on the left is shown with the posting user’s name shown above the “bubble”, again in smaller font and different color,
3. and at the bottom of the screen:
   • a text box in the middle,
   • an “AI” button to the left of the text box showing an “AI Star” icon. This button is enabled only when the text box is not empty and there is no networking session in progress,
   • a “Send” button on the right of the textbox showing a “paper plane” icon. This button also is enabled only when the text box is not empty and no networking session is in progress.

   When a button is “disabled”, it is grayed out and tapping on it has no effect.

While there is a networking session in progress, that is, while waiting for Ollama’s response on a rewrite or reply request, or while waiting for the sending of a chatt to complete, the “Send” button’s icon changes from showing a “paper plane” to showing an animated “loading” circle.

Front-end UX

As demonstrated in the video above:

• Upon loading, the Chatter front end retrieves all posted chatts from the chatterd back end and displays each according to the UI description above.

• User can enter text in the textbox and click the “Send” button to post a chatt to the back end.

• Instead of clicking the “Send” button after entering text, user can click the “AI” button to ask for a rewrite suggestion from Ollama, through the chatterd back end. The rewrite suggestion is displayed in the text box, replacing the original draft. Once the rewrite suggestion is fully displayed, the user can edit the text box, click the “Send” button to post the chatt currently in the text box, or click the “AI” button again for another rewrite.

• User can long-press on another user’s posted chatt displayed on the left of the timeline to request a draft reply from Ollama, through the chatterd back end. The reply draft will be shown in the text box at the bottom of the screen. Once the reply draft is fully displayed, the user can edit the text box, click the “Send” button to post the chatt currently in the text box, or click the “AI” button for a rewrite.

• Ollama’s response will be streamed from the back end. As each piece of the stream arrives, it must be immediately displayed on the text box, appended to what’s already there—instead of waiting for the full reply to complete.

Back-end infrastructures

The chatterd back end accepts HTTP/2 connection from the Chatter front end, with HTTPS security. The back end is set up with a self-signed certificate to provide HTTPS service. The back end is also set up with a PostgreSQL database to store posted chatts. The back end is further set up to forward user prompts to an Ollama server and returns Ollama’s reponse to the same user. Prompts to Ollama and the corresponding responses are not stored in the PostgreSQL database unless users subsequently posted Ollama’s responses as their chatts.

APIs

The chatterd back end has three API endpoint URLS:

• llmprompt: as documented in the llmPrompt tutorial, and
• postchatt and getchatts as documented in the Chatter tutorial.

Implementation guidelines

Back end

The chatterd back end is fully covered in the two tutorial specifications:

• llmPrompt: for setting up a AWS/GCP instance, installing an Ollama server, creating a self-signed certificate, and setting up an HTTPS server. It also provides implementation for the llmprompt API. You will also need to install your self-signed certificate on your front-end platform following the instructions in the first tutorial for Android or iOS.
• Chatter: for setting up a PostreSQL database. It provides the implemetation for both the postchatt and getchatts APIs.

Completing these two tutorials completes the back end. The back end you built for these two tutorials cumulatively can be used as is in this project. Any of the alternative back-end stacks provided will work. The host mada.eecs.umich.edu is not available for this programming assignment beyond the latter of the two deadlines of the tutorials. To receive full credit, your front end MUST work with your own back end.

Front end

The UI and UX for posting chatt on the Chatter front end, along with the front-end interactions with the postchatt and getchatts APIs, are all fully provided in the Chatter tutorial.

Front-end interaction with the llmprompt API is shown in the llmPrompt tutorial. You are to provide the rewrite and reply functionalities and UI/UX after you’ve integrated the two tutorials into the project. Step-by-step guidelines are provided below (follow the link to your platform of choice):

• llmDraft Android
• llmDraft iOS

server$ sudo systemctl restart chatterd

server$ sudo systemctl status chatterd