Finishing Touches and Deployment

With the project deadline approaching, the team applied the finishing touches to the Virtual Situation Room, as well as ensuring robustness throughout the entire system. While some members were working on deploying the app onto Azure, the following features were being finalized:

• Resilience testing for the agent orchestration pipeline
• Admin panel for role-based access to key endpoints
• "What's New" feature that enables report comparisons

Resilience Testing

Resilience testing is implemented within the agent orchestration pipeline to ensure consistent and stable report generation under varying conditions. Given the inherent non-determinism of LLM-based systems, these tests focus on validating robustness rather than exact output matching.

The resilience testing framework introduces controlled variations in input data, including formatting differences and reordered context. The pipeline is expected to produce semantically consistent reports despite these changes, ensuring that small input differences do not lead to significant deviations in final outputs.

The test suite also evaluates stability across multi-step agent interactions. By running extended rollouts of the orchestration process, the system is assessed for error accumulation, degradation in intermediate outputs, and its ability to recover from partially incorrect generations. This is particularly important for pipelines where outputs from one stage are recursively fed into subsequent steps.

Failure handling is explicitly tested through the injection of controlled faults, such as malformed LLM responses, missing intermediate data, or scoring edge cases. The pipeline is required to handle these scenarios gracefully, including retry mechanisms, fallback behaviours, and the ability to produce usable reports even when some components fail.

In parallel, deterministic helper functions—such as parsing, scoring, and data transformation utilities—are validated independently to ensure consistent behaviour across runs. This isolates variability to the LLM layer and prevents hidden sources of inconsistency within the pipeline itself.

The resilience testing effort contributed directly to stability improvements across the system. These include fixes to report generation inconsistencies, improved robustness in scoring logic, and more reliable handling of LLM interactions. Together, these enhancements ensure that the pipeline maintains reliable performance across a wide range of operational scenarios.

Admin Panel

Access to the admin panel is restricted to users with superuser status within Django’s built-in authentication system, ensuring that only authorised personnel can perform high-impact operations.

The admin panel provides a set of core management and operational functionalities:

• Crisis creation: Users can create new crisis entries through a structured interface backed by validated database models. This includes selecting a country from a predefined dropdown (covering 195 countries with ISO-standard codes), as well as providing key metadata such as event type and date via user-friendly input components (e.g., calendar pickers). These design choices reduce input errors and enforce consistency across records.

• Crisis management: Users are able to manage existing crises, including the deletion of entire crisis records or individual reports associated with a given crisis. The interface supports filtering and search capabilities (e.g., by country, event type, or date), enabling efficient navigation and management of large datasets.

• Data ingestion: For an existing crisis, users can initiate the automated data ingestion pipeline. This process retrieves and processes relevant external data, preparing it for downstream analysis. The integration with the admin interface allows ingestion to be triggered on demand without requiring direct system interaction.

• Report generation: Once ingested data is available, users can trigger the report generation pipeline. This produces structured outputs that populate the Virtual Situation Room dashboard for the selected crisis. Previously generated analyses can also be accessed and reviewed within the admin panel, enabling users to revisit historical outputs, compare results over time, and validate system behaviour.

In addition to these core features, the admin interface incorporates several usability and system-level enhancements. These include an audit trail (tracking who created or modified records and when), read-only historical fields for data integrity, and visual indicators such as colour-coded severity levels to support rapid assessment. The underlying database design enforces relationships through foreign keys (e.g., linking crises to countries), while indexing and validation mechanisms ensure both performance and data consistency. 

Crisis Creation Page

Report Analysis Page

So "What's New"?

After identifying the need for analysts to easily compare the latest report with previous ones, the “What’s New” feature was introduced. This feature compares successive reports and highlights key differences, with particular focus on changes in quantitative data that may indicate a worsening situation, as well as contextual developments that may require further investigation.

By surfacing these changes, the feature enables analysts to better understand how a humanitarian crisis evolves over time, rather than relying solely on static “snapshot” assessments from individual reports.

In addition to an agent-based approach, a non-intelligent comparison method was also implemented. This version performs a direct comparison between reports by identifying differences in the text, providing a straightforward way to surface changes without any contextual interpretation. While more limited, this approach offers a fast and deterministic baseline for comparison.

The more advanced version of the feature was implemented using a lightweight agent, which is provided with both reports as contextual input. The agent leverages its internal state management to effectively navigate and track the large volume of information across the documents, allowing it to maintain coherence while identifying meaningful changes. This enables a more structured and focused comparison, particularly when distinguishing between significant developments and less relevant variations.

Once a comparison has been generated, it is stored in the database for reuse in future queries. This enables a caching mechanism that avoids redundant computation when the same comparison is requested again, improving overall performance and reducing unnecessary processing overhead.

"What's New" component for Food Security->Food Availability

Azure Deployment

Due to the importance of scalability and high availability in the system’s design, the platform was deployed using Azure Web App services. As the application relies heavily on Azure-based component, such as Azure Database for PostgreSQL, Azure AI Foundry, and Azure Blob Storage, this approach simplifies long-term maintenance by consolidating all related resources within a single Resource Group. This centralised structure enables more efficient management, monitoring, and configuration of services. 

The backend, built with Django, is hosted on an Azure Web App, while the React frontend is deployed separately using Azure Static Web Apps. The two components are integrated through their respective endpoints, allowing seamless communication between the frontend and backend services while maintaining a clear separation of concerns.

To support scalability, the Azure Web App service can be scaled horizontally or vertically depending on demand, ensuring consistent performance under varying workloads. Additionally, deployment is streamlined through continuous integration and continuous deployment (CI/CD) pipelines, enabling automated builds and updates to both frontend and backend components.