Artificial intelligence is increasingly being considered as the next step in the evolution of Project Management Offices. The idea is attractive. An AI-enabled PMO could review project plans, identify risks, compare current projects with previous ones, recommend corrective actions and help management make better-informed decisions.
Imagine a project manager asking: Have we delivered a similar project? What went wrong? Which risks eventually became issues? What did we do about them, and did it work?
At portfolio level, the questions could be even more valuable. Why do certain projects repeatedly miss their deadlines? Which dependencies cause the most disruption? Are we continuing to underestimate activities? Have we encountered this supposedly “unexpected” problem before?
There is, however, an important prerequisite that is often overlooked: AI cannot learn from organizational experience that was never properly recorded.
I saw this firsthand some years ago while working with an organization that had deprioritized lessons learned and proper risk and issue management. The registers existed, but their purpose gradually became more about satisfying governance requirements than preserving useful knowledge.
Lessons-learned reports were completed at the end of projects to secure closure sign-off. Risks were often described in a few generic words, without explaining what might happen, why it might happen or how the project would be affected. Issues were not much better. In many cases, there was no clear description of the root cause, the impact, the action taken or whether that action had worked.
The organization later attempted to introduce an AI and large language model solution. It didn’t produce the value expected from it.
Technology itself was not the problem. The organization expected the LLM to learn from its previous projects, but those experiences had never been captured in a useful way. There was little substance behind the project records being fed into the solution.
This is why lessons learned, risk registers and issue registers should not be viewed as administrative documents. Together, they represent an organization’s project memory.
A well-written risk tells us what could happen, why it could happen and what effect it would have on the project. It also tells us who is responsible for managing it and what can be done to reduce either its likelihood or its impact. The Project Management Institute describes the risk register as a dynamic source of information about identified risks and the actions being taken to control them (link).
The word dynamic matters here. A risk register should reflect what is actually happening in the project. It should not sit untouched for several weeks and then receive a quick update the evening before a governance meeting.
An issue register deals with what has already happened. A useful entry explains how the issue emerged, what caused it, how it affected delivery, who dealt with it and what the eventual outcome was. Recording that an issue is “closed” tells us very little unless we also know how it was resolved.
Lessons learned should bring all of this together. They should capture both positive and negative experiences, then explain what another project should repeat, avoid or handle differently. PMI makes a simple but important point: lessons must be relevant and retrievable if they are to be effective (link).
When these records are maintained properly across several projects, they begin to tell a much bigger story. They can show that a particular dependency repeatedly causes delays, that certain projects consistently underestimate the effort required for project tasks, or that the same risk has materialized several times despite being described as unexpected.
They may also reveal that an organization has documented the same lesson repeatedly without ever changing its methodology, controls or way of working.
Historically, identifying such patterns required someone to manually review years of reports and registers. An LLM can make this knowledge much easier to access. However, it is important to be accurate about how this works.
In most cases, the organization would not continuously retrain the LLM each time a new risk, issue or lesson is recorded. It would more likely connect the model to its internal project information through retrieval-augmented generation, commonly known as RAG.
When a user asks a question, the system searches approved internal repositories, retrieves the most relevant information and provides it to the model as context. This enables the LLM to produce an answer based on the organization’s private information without that information having formed part of the model’s original training data (link).
For example, a project manager preparing for a major data migration could ask the system to review previous migration projects. The system might identify that business validation took longer than planned in four previous projects, that data ownership was unclear and that security approval became a critical dependency. It could then suggest mitigation actions that worked well in those projects.
If a new issue arises during delivery, the system could search for similar events, present their root causes and show how they were resolved. At portfolio level, it could identify patterns that may not be apparent from individual status reports.
This is where an AI-enabled PMO can offer real value. It can help the organization benefit from experience at scale instead of relying on individual project managers to remember what happened several years earlier.
However, the quality of its recommendations will remain dependent on the quality of the records it retrieves.
A risk described only as “vendor delay” provides little useful information. A stronger description would explain that there is a risk of the vendor missing the integration deadline because the technical specifications have not been approved, potentially delaying user acceptance testing and the planned go-live date.
Similarly, “data issue” does not tell the organization what happened, which information was affected, why the problem occurred, how it affected the project or what was done to resolve it.
Lessons such as “engage stakeholders earlier” or “improve communication” are also too general to be useful. Which stakeholders should have been engaged? At what point? What information did they need? What happened because they were not involved? What should the next project manager do differently?
An experienced person reading vague records may be able to make assumptions or ask follow-up questions. An LLM can only work with the information available to it. It may produce a well-written response, but that does not necessarily make the response accurate or useful.
The expression “garbage in, garbage out” is particularly relevant here. In fact, it is one of my colleague’s favorite phrases (you know yourself if you’re reading this). A highly capable model connected to weak project records will generate weak organizational intelligence, even if the response itself appears polished.
Local hosting addresses residency—not information quality
Data residency has frequently come up in my recent discussions with clients and colleagues, particularly in relation to government entities, banks and other regulated organizations in Saudi Arabia and across the region.
Saudi Arabia’s Personal Data Protection Law establishes obligations relating to the processing and protection of personal data (link). SDAIA has also issued guidance relating to the transfer or disclosure of personal data outside the Kingdom (link).
Depending on the information involved, applicable regulations and the organization’s risk appetite, an organization may decide to operate an LLM within its own infrastructure or through a controlled local environment. It is also technically possible to operate some models without allowing them direct access to the internet.
An LLM does not become useless simply because it cannot access the internet. It retains the language and reasoning capabilities gained during its original training. More importantly, it can still be connected to internal project systems, document repositories, policies, methodologies, audit findings and project records within the organization’s controlled environment.
I think of it as placing a capable analyst inside a secure records room. The analyst cannot browse the internet, but can read, compare and analyze everything in the room. If the records are detailed and reliable, the analyst can be extremely useful. If the room contains generic lessons, incomplete registers and carefully worded reports that avoid explaining what really happened, there is little to analyze.
Internet access can still be useful for current regulations, market developments, external benchmarks and other recent information. An organization operating an offline model would need a controlled process for periodically importing approved external material. Nevertheless, for many internal PMO use cases, the most valuable source of knowledge is not the public internet; it is the organization’s own delivery history.
Local hosting may address certain concerns relating to data location, access and confidentiality, but it does not solve the problem of poor information quality. An organization can invest heavily in infrastructure, security, model deployment and system integration and still receive disappointing results if its project knowledge is fragmented, generic or unreliable.
Enterprise Project Artifacts before AI
Organizations considering an AI solution for their PMO should start improving their information now.
Risks should be written clearly and should connect causes, uncertain events and potential effects. Issues should document root causes, decisions, actions and outcomes. Lessons should be captured throughout delivery rather than being reconstructed shortly before project closure. PMI has also advocated capturing lessons early and often rather than waiting until the end of the project (link).
More importantly, organizations need to create an environment in which project teams can document problems honestly. If registers are written mainly to reassure governance committees, the AI will receive an incomplete and overly optimistic version of reality.
I do not believe the real value of AI in project management lies in replacing project managers or making decisions for them. Its value is in making the organization’s accumulated experience available when someone needs it. It can remind a team that a similar risk has occurred before, explain how it was handled and perhaps prevent the organization from learning the same expensive lesson again.
So my question to you is: Have you recorded your experience well enough for anyone, human or artificial, to learn from it?