Why Projects Still Fail: Seeing the Real Bottlenecks in Digital Infrastructure Delivery Through the Agile Debate

Why Projects Still Fail: The Real Bottleneck in Digital Infrastructure Delivery, Seen Through the Agile Debate

In today’s era of ever-increasing infrastructure and digital investment, project failure is not uncommon. One notable signal is that even when organizations have already invested in delivery frameworks, transformation programs, methodology training, and tool platforms, IT projects still often fail to meet expectations. The discussion cited by iTnews attributes this problem to a deeper truth: the key to failure is not always the technology itself, but the execution system.

This may sound like an internal debate within the software industry, but its implications extend far beyond enterprise IT. Today’s port digitization, energy dispatch systems, rail ticketing platforms, public service cloud migrations, data center construction, and cross-department government service platforms increasingly resemble large-scale infrastructure projects: they too depend on complex coordination, long-term operations and maintenance, risk management, and organizational capability. Whether a project succeeds often depends on governance structures, capability chains, and implementation discipline, rather than simply on “which method was adopted.”

Methodology Is Not the Scarce Resource; Delivery Capability Is

A recurring judgment in this discussion is that Agile principles have not failed; what has really failed is the way organizations translate them into everyday delivery capability. In other words, the problem is usually not the framework, but how the framework is executed, understood, and continuously maintained.

This is especially important for the infrastructure sector. Whether building a new data center, upgrading a power grid dispatch system, or advancing a cross-city rail digital platform, project teams may have mature process maps, stage reviews, and supplier contracts. But if there is no alignment within the organization on goals, responsibilities, risks, and priorities, the project can still go off track.

From the perspective of engineering capital, this means many project failures are not caused by “having no method,” but by the following:

• a lack of shared definition of outcomes between leadership and execution teams;
• insufficient certification, training, and capability building, preventing methodologies from scaling;
• teams “adopting” a framework in form, but not translating it into results-oriented practice;
• lots of project management activity, but no effective visibility into real progress, value, and risk.

These issues are no stranger to large infrastructure projects. A port automation system, a power asset management platform, or a city-scale digital twin project will all drift off course if there are no clear governance boundaries and continuous feedback mechanisms. The result is often: “the system has gone live, but operations have not improved.”

“Doing Agile” Is Not the Same as “Being Agile”

One intriguing distinction in the discussion is that a team may be “doing Agile” without truly “being Agile.” This distinction applies not only to software development, but to any complex infrastructure system.

“Doing” often means completing a set of visible actions: daily stand-ups, retrospectives, iteration planning, task boards, release cycles. These actions are not wrong in themselves, but if they are not linked to business goals, delivery value, and risk control, they can easily become mere formalities.

In large infrastructure projects, this often shows up as follows:

• project plans still center on document compliance rather than capability delivery;
• suppliers and owners stay aligned on schedules, but fail to reach consensus on key decisions;
• teams are very “agile” at the local level, while the organization as a whole remains highly rigid;
• management believes the processes are in place, but overlooks the accumulation of deviations in actual execution.In large infrastructure projects, this often manifests as:

• Project plans still center on document compliance rather than capability delivery;
• Suppliers and owners remain aligned on the schedule, but have not reached consensus on key decisions;
• The team is very “agile” at the local level, but the organization as a whole remains highly rigid;
• Management believes the processes are in place, yet ignores the accumulation of deviations in actual execution.

This is also why an increasing number of digital infrastructure projects emphasize “end-to-end visibility.” Without visualized risks, value, and dependencies, complex projects can easily drift out of control while appearing to progress on the surface.

Leadership misalignment is the most common upstream cause of project failure

One of the most important conclusions from the discussion is that the root of project failure often lies not at the team’s end, but in leadership.

Many organizations are highly proactive when initiating transformation: setting a vision, approving budgets, introducing methods, and assembling teams. But once the project enters the execution phase, management withdraws too early, or focuses only on milestones while ignoring real systemic issues. This leaves the team to carry out complex tasks without clear boundaries or sustained support.

For infrastructure investment, this phenomenon is almost a breeding ground for cost escalation. Large projects are inherently capital-intensive, long in duration, highly interconnected, and supply-chain complex. Once upstream governance fails, subsequent issues are often not solvable by local fixes; instead, they propagate through the contract, design, delivery, operations, and financing structures layer by layer.

This is why many public infrastructure projects are placing increasing emphasis on:

• Whether the project governance structure is clear;
• Whether responsibilities are traceable;
• Whether performance is centered on “value” rather than “actions”;
• Whether risks are identified early;
• Whether there is genuine collaboration among the owner, contractors, consultants, and technical teams.

This is especially evident in digital infrastructure. Whether it is government cloud, smart city platforms, or data center expansion, if leadership treats the project merely as an IT procurement rather than a long-term operating asset, the probability of failure will rise significantly.

AI is not a cure-all; it is more like an amplifier

The discussion also touched on a realistic and urgent question: AI is entering delivery environments, but it may not fix organizational deficiencies—instead, it may amplify them.

On the positive side, AI can indeed reduce repetitive administrative burdens, such as generating draft documents, organizing meeting notes, and speeding up the preparation of user stories and delivery materials. This is valuable in any project environment with frequent collaboration, especially in digital transformation projects where staff are stretched thin and tasks are intensive.

But the problem is that AI does not replace judgment. It merely frees up some time, giving people the chance to make better judgments. If the organization itself lacks governance capability, has unclear data boundaries, and insufficient risk awareness, then AI will not bring automatic success; it will simply bring the problems to the forefront more quickly.

This is also the most important caution in the era of digital infrastructure:

• AI can improve document and communication efficiency, but it cannot replace project decision-making;
• AI can help summarize information, but it cannot guarantee the information itself is complete;
• AI can enhance delivery speed, but it cannot repair distorted organizational structures;
• AI can shorten certain processes, but it will amplify the data risks brought by governance gaps.- AI can improve document and communication efficiency, but it cannot replace project decision-making;
• AI can help summarize information, but it cannot guarantee that the information itself is complete;
• AI can enhance delivery speed, but it cannot fix a distorted organizational structure;
• AI can shorten certain processes, but it will amplify the data risks brought by governance gaps.

When enterprises embed AI into project management, customer service platforms, operations and maintenance systems, or public digital services, governance issues become even more prominent. AI itself is not the cause of project failure, but it may become an accelerator of existing weaknesses.

Implications for digital infrastructure investment: capability, governance, and operations are equally important

If we place this discussion back into a broader infrastructure perspective, the conclusion is actually very clear: in future digital infrastructure competition, it is no longer just about “who goes to the cloud first,” “who introduces AI first,” or “who launches a platform first,” but about who can deliver, operate, and iterate complex systems stably.

This is especially important for three types of investment.

First, data centers and cloud infrastructure. The value of these assets comes not only from racks and power capacity, but also from the delivery system, operations and maintenance system, and security governance. Loss of control during the construction phase will turn into ongoing costs during the operational phase.

Second, the digitalization of transportation and public services. Rail transit, ports, airports, and municipal services are increasingly dependent on data platforms and automation systems. If project management cannot coordinate across departments, digitalization will only increase complexity, rather than efficiency.

Third, large-scale transformation projects in government and enterprises. These projects are the most prone to the problem of “many frameworks, few results.” The more mature the methodology, the more the organization needs the ability to continuously learn and execute; otherwise, the system will become idle during scaling.

In this sense, agile is not the problem itself. The real problem is whether the organization has the structure to translate methodology into sustainable delivery capability.

Conclusion: In the infrastructure era, project success or failure depends on organizational engineering

In the past, people often attributed infrastructure failures to poor design, insufficient funding, or supply chain delays. Today, as digital systems become one of the core layers of infrastructure, a new failure pattern has emerged: projects do not lack tools; they lack organizational engineering capability.

This means that future infrastructure players with real competitiveness will not only know how to build, buy, and integrate, but also how to govern, learn, and continuously calibrate. Whether it is a data center, an energy platform, a port system, or a city’s digital foundation, what determines long-term returns is no longer just the scale of capital investment, but the quality of the execution system.

In other words, Agile does not “hurt” projects; what causes projects to deviate from expectations again and again is ignoring leadership coordination, capability building, governance transparency, and risk visibility.