Home

Appendix

Application Icon   AI in Practice

Capabilities

Now that we've discussed the technical aspects of AI, let's briefly cover where it's integrated into DEVONthink.

Chat Assistant: Many of your interactions will happen in the Chat inspector or Chat popover. Whether you're asking an impromptu question or chatting about the selected document, the assistant is made just for these things, all shown in a familiar message-style interface. And if you want to keep a record of the "conversation", you can easily save it as a reference for later. If you become a frequent AI user, you'll likely chat via the assistant often.

Document Summarization/Transformation: Summarize the document you're reading, or transform selected text via two buttons in the Navigation Bar.

Media and Image Processing: Process and transcribe audio and video files and examine images with machine learning. You can generate a transcript from audio or video files in your databases. Let AI determine the subject, transcribe text on signs, menus, etc. In both cases, the text is stored in one of a few ways. With one setting this can even happen automatically when adding the files to your database.

Document/Image Generation: Create documents with AI, both text-based and images. DEVONthink provides some AI templates made for creating text documents. And with the Data > New > Generate Image command and an appropriate AI model, describe an image and have it created for you. Additionally, some chat inquires can create new documents. You can read more about this in the AI and Your Documents section.

Search Syntax Assistance: When doing a toolbar search, an AI button lets you ask more naturally, e.g., "Show me PDF documents with more than 10 annotations." This shows the raw search syntax, kind:pdf md_annotationcount>10. You can copy this or just run the search.

Database Searching: You can ask AI to look for documents as you talk with the Chat assistant. For example, you could ask for it to find your espresso machine manual or a receipt for your telescope purchase. Fortunately, DEVONthink gives you a lot of control over what AI can access for searches. Check out the Safe Searching with AI section for more information.

Automation: Using chat queries and responses in smart actions like Chat – Query and the chat-based placeholder Query Response, present new automation possibilities for everyone. Additionally, there are new AI-driven commands available in AppleScript and JavaScript.

Practical Uses

The preceding section presents a more general view of where AI can be used. But let's look at some example use cases and methods:

AI Renaming Files: It's not uncommon to get a document with a less than useful name. Select it and tell the Chat assistant to Add the filename to the finder comments of this document then rename it with a reasonable name. This lets you preserve the original name, if needed, but also gives you something easier to find at-a-glance or in searches. Alternatively, you could tell it to Add only the filename as an alias on this document….

Another option is to use batch processing to rename selected files based on their content. These could be any type of file. Select a few documents and choose Rename to Chat Suggestion in the Tools > Batch Process submenu. Your default AI engine will attempt to examine each document and rename it. Note you can choose another engine by clicking the button.

AI Tagging Assistance: While it's possible to automatically tag documents on import, you may want to handle tagging on a case-by-case basis. Select a document and ask Chat, What tags would you recommend for this document?. If you like some of the suggestions but don't want to add specific ones, you could tell it to add them, e.g., Add three broad and two specific tags from that list. Or if this was in a recipe database, you could tell it to Add the ingredients as tags.

AI Documents from Web Content: If you have a bookmark selected in DEVONthink and want to gather specific information from it, you can ask Chat to create a document for you. For example, if you were reading about how to build a birdhouse, you could use Add the instructions on how to build a birdhouse on the selected page. You could also try a less specific prompt, like Capture the instructions on the selected page. The default format will be Markdown but you can request plain text, Markdown, or HTML documents from Chat. However, it is possible for the AI to respond with a format it decides.

Here's another example: Imagine you're shopping for a car and looking at a bookmark in DEVONthink. You get some results you like but just want to extract some specific information separately. Tell Chat to Capture the two cheapest cars newer than 2014 with less than 100,000 miles from this page. With some AI engines, you may get the document and also some commentary about them as well.

AI Searches: With AI, you can do "semantic searches", essentially looking for things related by concept instead of specific terms. As an example, ask Chat to look for documents that talk about vehicles and watch it generate its own query, e.g, searching for car* OR boat* or plane*…. Logically, you can be more specific, as needed. In a database of PDFs from car manufacturers, you can ask for documents about Italian vehicles without mentioning a brand, and it will return documents about or that mention Italian vehicles.

One thing you may be surprised by is how natural a chat can be. Say you're gathering information for a local animal shelter and want to locate some of your documents on different breeds. You ask Chat, What documents discuss Manx cats?. You then ask, What about collies?. Then you ask very naturally, ferrets?, without repeating the full question, and get results. Using an AI with a large context window, the previous discussion is considered when replying to subsequent questions.

Also see the Safe Searching with AI section about how to have some control over AI's reach.

Get to know your AI

As AI engines aren't all the same, what you ask about and how you phrase things can vary. Also, their capabilites depend highly on the selection and the settings. So how do you know what is possible and how to talk to your AI engine? Open the Chat inspector or Chat popover and ask these important questions (depending on the responses you receive):

  • Icon
    Create a table of your capabilities: This produces a list of what the AI can do within DEVONthink. You may be surprised at what it is capable of. This one is the most important to get a real understanding of what's possible.
  • Icon
    Create a table of the properties you can get and set: If the AI can access or modify the properties of an item, it's good to know which ones.
  • Icon
    How can I effectively ask you to search for something?: It's important to know the terms and phrasing an AI model understands. Asked this way, it usually provides examples as well.
  • Icon
    How can I effectively tell you to create an item?: Following suit, you can determine how best to have AI create items, if it reported it can.
  • Icon
    How can I effectively tell you to update an item: If the AI reports it can update existing documents, it's good to know this phrasing too.

These questions can be especially useful if you're trying to decide which engine to use. Ask your questions, choose a different model or engine, and ask it the same ones. For future reference, you can save the chat or even tell the assistant to put its responses in a new document.

Another handy tip if you're testing, or even just curious about, how models respond to the same prompt. Choose a model and use it as your normally would. Then choose a different model or AI engine in the popup menu. Now enter Same prompt and the newly selected engine will respond to the last prompt you gave to the previous model. This can be useful when you're testing responses from specific models or AI settings.

We hope these last two sections gave you a clearer understanding of the complementary, but powerful, role of external AI in DEVONthink. You'll find other passages threaded throughout other chapters, e.g., in Automation. As it is a feature of the higher editions, keep your eyes open for blue sidebars or sections of blue text as you read.