Project 3: llmAction SwitUI

Cover Page

Preparing your GitHub repo

If you have not completed the llmTools and Signin tutorials, please complete them first. We will base this project on the llmTools code base, but will be copying over the bulk of the Signin code also.

In the following, replace /YOUR:TUTORIALS/ with the name of your tutorials folder.

• On your laptop, navigate to /YOUR:TUTORIALS/
• Create a zip of your llmtools folder
• Rename your llmtools folder llmaction
• Prepare your repo as you have done in previous tutorials and projects
• Push your local /YOUR:TUTORIALS/ repo to GitHub and make sure there’re no git issues:
  git push

  • Open GitHub Desktop and click on Current Repository on the top left of the interface
  • Click on your reactive GitHub repo
  • Add Summary to your changes and click Commit to main
  • If you have pushed other changes to your Git repo, click Pull Origin to synch up the clone on your laptop
  • Finally click on Push Origin to push changes to GitHub

Go to the GitHub website to confirm that your folders follow this structure outline:

  reactive
    |-- chatterd
    |-- chatterd.crt
    |-- llmtools
    |-- llmaction
        |-- swiftUIChatter
            |-- swiftUIChatter.xcodeproj
            |-- swiftUIChatter
    |-- tools            
    # and other files or folders

Obtaining chatterID

The main purpose of the get_auth tool is for the LLM to obtain an authorization token. We require LLMs to acquire an authorization token to make operative tool calls that can take action with real-world side-effects. The authorization token requirement forms our human-in-the-loop guardrail. In this project, the chatterID first used in the Signin tutorial is our authorization token. To create a chatterID requires user to sign in using Google Signin. To store/load the chatterID for reuse within its limited lifetime, we further require biometric authentication from the user. Both Google Signin and biometric check requirements mean that the authorization token can only be obtained from user-facing front-end devices.

The front-end UI/UX and the network streaming and tool call infrastrucure are those of llmTools. To these, we add code from the Signin tutorial to use Google Signin and biometric authentication.

Signin migration

• Follow the instructions in the Signin tutorial to install the Google Signin SDK. Then copy over the OAuth client ID (and the reversed client ID) created. You can review the instructions in the tutorial on how to create these, especially steps 3-8 of the second set of steps in the linked section. You wil also need to provide justification for performing biometric check by following step 1 of the biometric check section.

• Copy over code from the Signin tutorial that

  • handles signing in to Google,
  • uses the idToken obtained from Google Signin to further obtain a chatterID from the chatterd back end,
  • saves and loads chatterID to/from secure storage with biometric authentication, and
  • updates to the rest of the application needed to support Signin.

ChattViewModel

Recall that the GoogleSignIn SDK pops up a UIKit window. We launch it by toggling getSignedin and providing signinCompletion, both properties of ChattViewModel. Since we need to access ChattViewModel from the networking code only for chatterID-related operations, we make ChattViewModel available to the ChatterID singleton:

• add a vm property to ChatterID, of type optional ChattViewModel and intialized to nil,
• update the init() block in your swiftUIChatterApp to:

    init() {
        LocManager.shared.startUpdates()
        ChatterID.shared.vm = viewModel
      
        Task.immediate { [vm = viewModel] in
            await ChatterID.shared.open(errMsg: Bindable(vm).errMsg, showOk: Bindable(vm).showOk)
  
            if let appID = vm.appID, !vm.sysmsg.isEmpty {
                await ChattStore.shared.llmPrep(
                    appID: appID,
                    chatt: Chatt(name: vm.onTrailingEnd, message: vm.sysmsg),
                    errMsg: Bindable(vm).errMsg)
            }
  
            vm.showError = !(vm.showOk || vm.errMsg.isEmpty)
        }
    }
  

• update the sysmsg in ChattViewModel to say:

    let sysmsg = "Use pull to add a model, rm to remove or delete one, and ls to list or show
    models---always provide an empty string '' as the argument arg to ls. Show listing as a table.
    Use get_auth to get authorization token before each call to ollama_cli." 
  

  You can also update the initial message in ChattViewModel to say:

    var message = "List Ollama models."
  

Toolbox

Assuming you have created the get_auth JSON schema file in your tools folder, using the getAuth() function we will define in the next subsection, add an entry to the TOOLBOX switch table to register the get_auth tool. If you’re not sure how to do this, you can use the entry for get_location in the TOOLBOX switch table from the llmTools tutorial as example.

getAuth() function

The main purpose of the getAuth() function is to return a valid authorization token to the LLM. A “valid” authorization token in this project is a chatterID that has not expired. Implement the function getAuth(). The function first checks if there’s a valid chatterID. If so, it returns the chatterID. Otherwise, it launches SigninView to let user sign in with Google Signin. Repeat these steps until there’s a valid chatterID.

If sign in is successful, there should now be a valid chatterID we can return to Ollama in the form of a new prompt. Otherwise, we *must return HTTP status code “401: Unauthorized” as the tool call result to Ollama. Ollama will inform the user that authorization has failed in its prompt completion.

Consult how SubmitButton in the Signin tutorial accomplishes both of the above and adapt it to implement getAuth() accordingly.

With that, you’re done with the llmAction project! Congratulations!

Run and test to verify and debug

Support for Ollama 0.20.2

Ollama on mada.eecs.umich.edu was recently upgraded to support gemma4 models. Ollama version 0.20.2 puts the thinking output of models on a separate JSON field, thinking. To support this new version, please make the following changes to your ChattStore.swift:

struct OllamaMessage: Codable {
    ...
    let thinking: String?
    ...
    
    enum CodingKeys: String, CodingKey {
        ...
        case thinking = "thinking"
        ...
    }
}

Once in llmPrep(appID:chatt:errMsg:showOk:) and twice in llmTools(appID:chatterrMsg:) when instantiating the OllamaMessage struct, set thinking: nil.

When parsing an SSE data line, in addition to accummulating tokens in the content field of a message, accummulate tokens in the thinking field of message also:

                        if let token = ollamaResponse.message.content, !token.isEmpty {
                            // ... keep existing code
                        } else if let token = ollamaResponse.message.thinking, !token.isEmpty {
                            resChatt.message?.append(token)
                        }

The updated code should continue to work with older versions of Ollama.

Please see the End-to-end testing section of the spec to test your front-end implementation.

Once you finished testing, change your serverUrl back to YOUR_SERVER_IP so that we know what your server IP is. You will not get full credit if your front end is not set up to work with your back end!

Front-end submission guidelines

Unlike in previous tutorials and projects, there is one CRUCIAL extra step to do before you push your lab to GitHub: ensure that the Bundle identifier under the Signing & Capabilities tab of your Project pane is the one you used to create your OAuth client ID. Otherwise we won’t be able to run your app.

Be sure you have submitted your modified back end in addition to submitting your updated front end. As usual, we will only grade files committed to the main branch. If you use multiple branches, please merge them all to the main branch for submission.

Push your front-end code to the same GitHub repo you’ve submitted your back-end code:

• Open GitHub Desktop and click on Current Repository on the top left of the interface
• Click on the GitHub repo you created at the start of this tutorial
• Add Summary to your changes and click Commit to main at the bottom of the left pane
• If you have pushed code to your repo, click Pull Origin to synch up the repo on your laptop
• Finally click Push Origin to push all changes to GitHub

Go to the GitHub website to confirm that your front-end files have been uploaded to your GitHub repo under the folder llmaction. Confirm that your repo has a folder structure outline similar to the following. If your folder structure is not as outlined, our script will not pick up your submission and, further, you may have problems getting started on latter tutorials. There could be other files or folders in your local folder not listed below, don’t delete them. As long as you have installed the course .gitignore as per the instructions in Preparing GitHub for Reactive Tutorials, only files needed for grading will be pushed to GitHub.

  reactive
    |-- Prompts.txt
    |-- Rules.txt
    |-- Skills.txt
    |-- chatterd
    |-- chatterd.crt
    |-- llmchat
    |-- llmaction
        |-- swiftUIChatter
            |-- swiftUIChatter.xcodeproj
            |-- swiftUIChatter
    |-- tools            
    # and other files or folders