To clarify, an AI model is the technology used by services like ChatGPT, etc. In contrast, DEVONthink's AI predates these LLMs by 20 years and was built by us using an entirely different set of methods, methods that are still as powerful and functional today as they've always been. For example, the See Also inspector, the Tags and Graph inspectors, as well as commands in the Data menu are still controlled by our internal AI. The remainder of this section is about external AI services, like ChatGPT.

If you're new to AI, perhaps having heard people talking on forums, the news, etc., but you don't know much about it, here's a very simplified intro.

How does chat work?: At a very simplified level, your chat questions are broken down into bits and pieces, called "tokens", which are processed by AI. It examines a token then looks into its huge database of parameters trying to identify it and the most likely token that would follow it. It puts the best match in place then moves on to the next token. This process is repeated over and over again, mathematically constructing the tokens into words, then sentences returned to you as a response.

Parts of a query: There are three parts to making an AI query. Two are mandatory; the third, optional but useful.

We have also included some relevant terms in the Glossary > AI section of the Appendix.

Choosing an LLM: Remember that using AI is entirely optional. However, since you're reading this you are at least curious about it. So the first step is choosing an LLM. If you look at the AI > Chat settings, you will see where you set a default Chat model. These are the currently supported options, with no specific advocacy for any:

As we've mentioned, adding AI capabilities was no trivial thing to investigate, discuss, and implement. And while we have come to terms with the areas of concern we could control, on a personal (or professional) level, there are still topics that you must consider and decide on for yourself.

Privacy: We are firm believers in data-privacy and we do whatever we can to keep things in your control. However, when dealing with a commercial AI service, your questions and potentially documents go to servers controlled by that service. We don't advocate being paranoid about using such a service, but it's good to know about and feel comfortable with whom you are sharing some of your data. However, you may wish to opt-out of having your AI "conversations" being used to further train their AI.

DEVONthink doesn't allow AI to have uncontrolled access to your data and databases but takes several steps to safeguard your privacy. By default, using the Chat assistant or any AI assisted automation only uses selected documents. When the database search option is enabled in the settings, only the selected items, items matched by smart rules, or the group selected in the Navigate sidebar, i.e., the current location, are accessible to AI, further limiting its reach.

DEVONthink also never sends your original document to an AI engine, taking these steps to keep things private:

Lastly, DEVONthink doesn't come with AI access enabled and running. It's up to you to set up and choose the options you want to use. So using external AI is completely optional, not a requirement, and not using it may be exactly the level of privacy you want.

Expenses: "There's no such thing as a free lunch." This applies to using a commercial LLM as well. While they typically offer a free account of some kind, it's very limited and made for familiarizing yourself with the process and responses. Once you determine this is something you want to use more often, get out your credit card. You will be purchasing tokens, again "bits of words", either in bulk or running up a tab as they're used. While they're typically relatively inexpensive per-token, heavier use of AI will deplete your reserves or increase your bill faster. As a courtesy, we've included property icons in the AI settings denoting if a particular chat model is known to be expensive.

Another thing to be aware of is the difference between using a chat agent, e.g., talking to ChatGPT, and using their API. An API is how third-party applications programmatically access services and data provided by a company. To use commercial AI services, you will need to create an API key for the service and enter it in the Chat > API Key settings. Check with your AI provider how to generate a key and any involved costs.

Here are some general recommendations that may help curb costs:

Quality of results: As we discussed previously, these chat engines are doing incredible computational gymnastics to produce responses but don't have any actual knowledge to draw from. It has no way to verify its answers in the way a human can. Earlier on they were known for "hallucinating", i.e., returning responses that were complete sentences but made no sense or had little to no relevance to the question asked. Things have certainly improved, and likely will continue to, but it is still possible to have incomplete or inaccurate responses. So while you may get a reasonable response, be aware the result isn't guaranteed to be accurate. Especially on questions of consequence, you should be checking the responses.

The limitations of local AI: No one wants to share their data. No want wants to pay for AI. We certainly understand those things. While there is an ever-growing list of "Run the AI of your choice on your Mac now!!" applications, let's take a realistic look at running your own AI:

However, if you're inclined and curious — and your hardware can support it — using local AI is certainly something you can explore.

While there are many things that can be done, there are still limits imposed by privacy concerns or technical considerations. For example, AI has access to the current location in a main window, not all your databases. It also isn't going to operate like an automaton, creating databases, constructing its group structures, downloading and filling those groups, then examining and issuing reports on it all. It is an "assistant" in your labors, not your replacement.

Another critical thing to be aware of, AI is not going to "process and connect" years of your documents and information in your database. The way AI is hyped by many makes this sound feasible, but it actually is not. Will you be able to process documents in a useful way? Of course, but on a much more limited scale. So while we understand the hope, actually accomplishing this would be time and cost-prohibitive for most people and require sharing all those documents with third-parties.