Generative AI can generate data from science projects to birthday party ideas, a wide range of generation. These generated content is derived from either a learning data from which the model has been trained on or from user feedback and conversation context provided during model to user conversation.

Learned Data

Models are first trained on sets of data on different topics so they can learn and analyze these data which helps them further in generating their own data from these samples. So any model first goes in learning mode.

Learning from Training Data

Specialised set of sample data which is fed to the model in the learning mode which creates a context memory of the model.

Learning from Publicly Available Data

Model learns itself by analyzing data available from Public Space and as it is finds more and more data accuracy of the model improves.

These learnings of the data either from training sets or publicly available data builds model context, which is like a memory for the model. The higher the context means more memory the model has and better generated data you can get. Claude has 1M context.

User Feedback

When the user enters a prompt, the model breaks down the prompt into intents and understand the requirements, it passes the requirement through its context memory as well as publicly available data and a combination of these generates new content.

Now the content can be either meeting the user expectation or may not, so based on user feedback the model keeps regressing the data and updating its context which in time enhances the accuracy of the model.

Limitations

Hallucinations

Models are trained periodically on training sets, which means a model always has a past data or knowledge. Lets say the model was trained last on 30th May 2026, then the model has data which it gathered till that day. This is called Knowledge cut-off date.

Now lets say someone asks a model to find who won the cricket final this year on June 1st 2026 for a match which was played on 31st May 2026 for which model may not have upto data or knowledge in its context, so what it will do is break the prompt into intents -

Prompt: Who won the the cricket final this year?

Intent: cricket final winner this year

Model Context: India won cricket final in 2025, however it may be India did not win the cricket final this year but model returned the data as per the context. This limitation is called hallucination. So AI can give incorrect output with full confidence as well.

Non-Deterministic Output

Model decides what data comes next based on user input, for example if you keep asking the same question to the model multiple times, it may get confused if the question has ambiguity.

For ex - Who won world cup this year? Now in this prompt we have not specified world cup of which sport - cricket, football, etc.

So the model based on its conversational context and user interest behavior automatically chooses a sport lets say cricket and gives the answer that Australia won the world cup this year. But if you ask again then model may get confused and this time it will give the answer related to a different sport as the user feedback from earlier response would be considered negative and non-accepted answer of model.

How to resolve these limitations

Interactions of Human and AI - Human in the loop. Humans are there to provide critical thinking, judgement, creativity and ethical oversight to AI which keeps improving its context to generate better and accurate data.