From behind the scenes
And those were only the main principles.
Connecting an AI model and letting it talk is the beginning. Then comes everyday use — and with it a great many small decisions that determine whether the assistant actually helps you.
Here are 100 selected questions we worked through while building Darwin. From the first click on the microphone to the situations where several agents are working, the connection drops, or an update arrives.
01Speaking and listening10
- Do you have to hold a button down while you speak, or is one click to start and another to stop enough?
- What should happen when you start speaking while the assistant is still answering?
- Does interrupting also stop audio that is already queued for playback?
- How does the assistant tell a pause mid-sentence from a finished instruction?
- How do you stop the assistant from listening to and processing its own voice?
- What happens when the microphone picks up only silence or noise?
- Can a recording in progress be cancelled without being sent?
- Should the space bar switch on the microphone even while you are typing into a form?
- When should listening resume after an answer?
- How does the user tell whether the assistant is listening, thinking, or already answering?
02When you combine voice, text and attachments10
- Can speech simply be dictated into the text field and corrected before sending?
- Does further dictation append to the text already written, or overwrite it?
- What gets sent when you have text typed in the field and say the rest of the instruction out loud?
- Can you type an exact file path and speak the instruction that goes with it?
- Does the text you had written come back into the field if processing the voice instruction fails?
- Can a screenshot be pasted straight from the clipboard without saving it to a file?
- Can a document be added by dragging it into the window?
- Can you attach several files to a single instruction?
- Does a prepared attachment go along with a spoken instruction as well?
- How does the user know an attachment has already been sent and is not waiting for a further instruction?
03Answers, voice and everyday control10
- Can you work with the assistant completely silently?
- Does silent mode also switch off voice generation, or does it only mute the speakers?
- Can an answer already shown as text be read aloud afterwards?
- Will short progress notices and the final answer play back in the right order?
- Can agents use different voices without talking over one another?
- Can the language of the interface differ from the language of the answers?
- How does the phrasing change when a male voice is switched to a female one?
- Do the chosen voice, language and mode survive a restart?
- What happens when the voice service fails but the text answer is already finished?
- Can the assistant's face be shown on a second monitor without the voice playing twice?
04When you walk away or the connection drops10
- Does access to sensitive functions lock when you leave the assistant unattended?
- Does an open window automatically mean permission to work with files and e-mail?
- What does an unknown device see when it opens your assistant's address?
- What happens to a task in progress when the phone screen locks?
- Does the assistant finish the answer even after the connection is interrupted?
- Do you get the result once you reconnect?
- Does a delivered answer appear twice if it was already loaded from history?
- Should paid voice be generated when no listener is connected any more?
- What happens when you have the assistant open in several tabs?
- Does an instruction from the phone open a page on the remote computer, or offer a link on the phone itself?
05Memory and your own documents10
- What is the difference between the conversation history and information the assistant should remember long term?
- How does the assistant find a note when you do not remember its exact name?
- How does it pick the relevant memory without loading the whole archive into every question?
- How do you stop thousands of documents from burying a handful of important personal notes?
- Can the search be narrowed to a particular project or collection?
- How does a change to the original document show up in search?
- What happens to the index when you delete the original file?
- How do you stop the same document from appearing in results through several copies?
- What exactly should be removed when the user says "forget this"?
- How do you tell your own memory from information delivered by another assistant?
06Agents and dividing the work10
- Can the user decide which agent an instruction belongs to?
- How is an agent chosen when the user never mentions its name?
- Can each agent have its own brain, voice and working resources?
- What if the main assistant works but one agent's brain is refusing requests?
- Can work be handed to another agent while the first one is still working?
- Can one agent be stopped without interrupting the others?
- How do you stop concurrent agents from mixing up attachments and context?
- Where does an agent store the result and what it learned from its work?
- How does the user tell an instruction an agent accepted from work that is genuinely finished?
- What happens to the tools of an agent mid-task when the backend restarts in the meantime?
07Processes and automations10
- What triggers a process — the user, the clock, a new message, or a change in a folder?
- Can a process watch several working folders at once?
- Should a new folder appearing count as an event the same way a new file does?
- How do you stop a process from running on a file that is still being copied?
- Should the process handle every new file separately, or wait for the whole batch?
- Does folder watching remember its state across a restart?
- What happens when the watched drive is temporarily unavailable?
- Does the process receive the specific new material, or only a general notice that something changed?
- How should a process behave after being paused and switched on again?
- How do you stop one process finishing from starting an endless loop of others?
08Tools, plugins and connected services10
- Does an installed plugin also mean a correctly configured, usable function?
- How does the assistant learn which operations a new plugin supports?
- What happens when one plugin has a faulty tool definition?
- Does the same capability still work after switching to a different brain?
- Do the same permissions apply to a spoken instruction and to a click on the plugin's page?
- Does an agent named "Accountant" also mean a real connection to an accounting application?
- How does the assistant tell a prepared e-mail from a message already sent?
- How is it verified that the user approved exactly the version of the message that is about to go out?
- Can the assistant tell an estimate from an actual run before a paid media operation?
- How does the user find out whether the problem was the licence, a permission, a missing setting, or an external service?
09Updates and long-term use10
- Does the user keep their settings, memory and agents after an update?
- What happens to a plugin the user modified themselves?
- How is a fix to a built-in plugin delivered without losing custom extensions?
- Does a new function arrive together with the libraries it needs?
- What happens when a new interface page loads with old files still in the browser's cache?
- Can the user find out which step of the work failed after an error?
- Does the error message distinguish a temporary service outage from a wrong setting?
- How do you stop credentials from ending up in diagnostics or in the assistant's answer?
- Does user data survive the end of the trial version?
- How do you restore a working installation when an update does not go through cleanly?
10And other details outside the conversation itself10
- How does the assistant tell two similarly named events apart when you want to move one of them in the calendar?
- Which account should it read mail from when you have both a work and a private mailbox connected?
- Can the right camera be chosen when several devices are connected to the computer?
- How do you stop ordinary hand movement during speech from triggering control gestures?
- Should a gesture change the size of the assistant's face, or the zoom of the memory map that is open?
- How is a particular light matched to the right room in the model of the house?
- Will the house view work even without temperature sensors connected?
- What happens to a task handed to a colleague when their Darwin is not available right now?
- Should a translated video keep the original voice and timing, or create a new recording of the translated text?
- Can you find out, before processing a recording, how much of it is speech and how much paid time would be silence alone?
And even this is only a fraction.
These are only 100 selected questions, not a complete list of what Darwin contains. The calendar, gesture control, the camera, the house and its devices, working as a team, video processing and other areas that are already finished have their own settings, connections and dozens of similar details. On top of that, features meet: while an agent is working another instruction arrives, a setting changes, or a connected service drops out — and even then it has to be clear how the assistant should behave.
Behind every such question is a decision about behaviour, several parts wired together, and checking what happens outside the ideal case. We work on Darwin every day: extending what it can do, tuning everyday use, and working through further situations from real life. That work is what produces an assistant you can actually use every day.
Building with AI has its own overhead too.
On a small prototype a handful of instructions and a basic subscription may be enough. Once the application grows, though, even a short "fix this behaviour" can mean reading several parts of the project, working out the connections, making changes and checking them again. Difficulty therefore cannot be counted by the number of messages alone: it also depends on the model, the size of the context, and the work the instruction sets off. How consumption differs on development tasks.
Part of the work is how we give the AI its tasks and what material we prepare for it. One thread for every unrelated problem gradually accumulates history nobody needs; starting every small change completely from scratch means explaining the project over and over. It helps to keep related work together, to separate distinct topics, and to capture decisions as you go, so that picking up again does not mean working everything out afresh.
Caching can play a part too — reusing the already processed shared part of the input. On APIs that support it, this can cut both processing time and the cost of repeated context, but whether it applies depends on the content matching, the model, and the provider's rules. Continuous work can therefore, under certain conditions, make better use of the cache than returning after long breaks; the length of the break alone, however, does not determine consumption or price. And the rules of an API cache do not in themselves explain how a particular subscription allowance is counted down. OpenAI: input caching, Anthropic: input caching.
How the code is organised matters in the same way. Splitting it by responsibility into clear modules helps both a person and an AI work with the relevant part instead of going through one enormous file again and again. A higher number of files guarantees nothing by itself: what decides it is clear boundaries, understandable connections, and being able to verify a change without damaging the other functions. Darwin, as it grows, keeps showing us where the original arrangement needs splitting further.
Even with careful instructions and well-organised work, during intensive development we find ourselves using up the weekly allowance of a higher subscription at roughly the halfway point. That is our experience from building this, not a rule for every project. So behind a finished assistant there is also the time spent on working with context, on testing and on maintenance — not only on generating code.
Want to try it from the other side?
If you are building your own, take our four prompts. If you would rather see the finished thing, ask for a trial — same page.
Free prompts › Meet Darwin ›