Build an AI That Only Uses Sources You Trust

AI is usually good at giving you more information. Sometimes too much information. The harder problem is making sure it is pulling from the right information. 

I have a friend who told me her muffins tasted like ChatGPT muffins. I found this hilarious, but I also knew exactly what she meant. A recipe pulled from all the muffin recipes in the world can end up being a bit bland. There are a lot of bad recipes in the training data, and the model does not necessarily know which ones you would trust.

It made me think about how often we ask AI to give us the best answer without first telling it where the answer should come from.

This week, you are going to build an AI assistant on top of a small knowledge base you create yourself.

You could make a recipe assistant that only uses cooking sites you trust, a research assistant grounded in a specific set of reports, or a sales assistant that only uses your company’s messaging and case studies.

The point is to give the AI a clear set of sources and see how well it can work within those boundaries.

The Challenge:

Pick one narrow job for your assistant and decide which sources it should be allowed to use.

Then build a version that searches those sources before answering, shows where the answer came from, and tells you when the information is not available.

You do not need to know which tool to use before you start. Some people may use NotebookLM, a custom GPT, or a Claude Project. Others may decide to build something more advanced. The prompt below will help you compare the options and choose the simplest one that works.

The prompt to get you started

Drop this into your AI of choice to get started:

I want to build an AI assistant that answers questions using only sources I trust.

It should search the approved sources before answering, show where each answer came from, say when the information is not available, and avoid filling gaps with general knowledge.

The assistant I want to build is:

[Describe what you want it to do.]

The sources I want it to use are:

[List the websites, documents, notes, or files.]

Help me compare a few ways to build this, from simple no-code tools to more advanced options. Recommend the best approach for my use case, then guide me through building and testing it.

Test it with one easy question, one question that combines multiple sources, and one question it should refuse to answer.

Too Easy:

Once the assistant is working, build a process for keeping its knowledge base current by adding new approved sources, removing outdated information, or flagging places where your sources disagree.

You can also make the system stricter by requiring a source for every answer. If it cannot find one, it should not answer.

Try this prompt:

Help me improve this assistant so its knowledge base stays accurate and up to date.

Create a simple process for adding new approved sources, reviewing or removing outdated information, and identifying conflicting sources.

Then help me strengthen the rules so every answer must include a source. If the assistant cannot find enough evidence in the knowledge base, it should clearly say that it cannot answer.

Why this matters?

A lot of AI work focuses on improving the output, but the quality of an answer also depends on the information the model has to work with.

When you create a trusted knowledge base, you are giving the AI better context and clearer boundaries. You are deciding where it should look before it responds. A better assistant is one that knows where you would look and helps you find what you need faster.

That’s it for this week.

Hit reply and tell me what you thought of the challenge this week.

I read every response, and your experience helps me shape these challenges. If you get stuck or have questions, reach me at hello@tinyailab.ai