Section 7 — Summary and Design Principles: The Final Checklist for an Architect
The six previous sections were a journey. They started with the reason scattered logs fail and moved through the members of the tool family, the pipeline that carries events, the SIEM that reasons about them, and the operational challenges that follow. The last of them put those challenges into complete scenarios.
This final section turns that journey into an executable checklist. The goal is not to repeat what came before, but to extract the principles an architect can build a design on from the first day. It also names the mistakes that sink most central logging projects.
Ten Design Principles for Central Log Architecture
These ten principles came out of the previous sections. Each one is a design decision, and the cost of that decision depends almost entirely on when you take it. Taken on day one it is a line in a diagram, and taken after deployment it is a rebuild.
Centrality, Not Absolute Concentration
Centrality means a single source of truth for analysis, not that all data sits on one server. A good architecture can have several collection layers, a relay in each data centre and a central aggregator, and still present one unified view to the analyst. Absolute concentration, meaning everything on one machine, creates a single point of failure that is both a security problem and an operational one. Centrality belongs in the analytical view, not necessarily in the physical storage layout.
One Clock, a Prerequisite Rather Than an Option
Without NTP, no analysis is trustworthy. This principle is fundamental enough that it has to be implemented before anything else. ISO 27001 control 8.17 explicitly requires clock synchronisation to a single time source. If you do not have NTP, the rest of your architecture stands on shifting sand.
NTP itself has to be secure as well: restricted, monitored, and authenticated where the protocol allows it. An attacker who can move the clock corrupts the evidence that everything else depends on.
NTP is a prerequisite rather than an option, and it sits at the top of the roadmap. Establish a single time source before connecting a single log source, because every layer above it depends on the ordering it produces.
A Correlation ID, From Day One
In a distributed system, reconstructing a request that crossed several services is impossible without one shared identifier carried through every log line. That identifier has to be designed into the applications from the first day, not added later as an improvement. If your applications do not emit one today, it is a technical debt item and it belongs at the top of the list.
Layering Rather Than One Piece
The log pipeline is one connected system, but it is built from distinct layers: generation, transport, collection, processing, storage, analysis, and response. Each layer has to be independently configurable, monitorable, and replaceable. If you replace one layer such as storage, you should not have to rebuild the others. That separation is what buys flexibility and maintainability.
Buffering in Several Layers
A network failure can happen at any point on the path: between the source and the relay, between the relay and the aggregator, or between the aggregator and storage. A buffer at one layer is not enough. You need buffering at the edge in the agent, and in the centre in the relay or aggregator.
Buffers also have to be monitored, because a full queue discards data silently. Size them for the longest outage you are willing to tolerate, not for the free disk space you happen to have.
The Data Lifecycle, From the Start
Retention is an economic and legal decision rather than a technical one. You should decide at the beginning what data you keep, for how long, why, and in which tier, whether Hot, Warm, Cold, or Frozen. PCI-DSS requires at least 12 months of retention with 3 months immediately available, and HIPAA and SOC 2 carry their own requirements. Designing the lifecycle after deployment is slow and expensive.
One point deserves emphasis: old data has to stay searchable, or post-incident analysis becomes impossible at the moment you need it most.
Least Privilege, Even in the Log System
A central log system is a high-value target: if it is compromised, you lose everything at once. Access follows the principle of least privilege, which means nobody reaches every dataset. Sensitive data should be separated or masked, and logs should be kept in immutable storage so that an attacker cannot alter the evidence. The security of the logging system matters as much as its analytical capability.
Restrict read access by role, keep sensitive fields separated or masked, and store security logs immutably. An attacker with write access to the evidence destroys its value entirely, and no later investigation can prove what actually happened.
Privacy by Design
Logs contain personal data: usernames, IP addresses, email addresses, and sometimes more. GDPR and similar laws protect that data, and three rules apply from the first day. Collect only what you need, mask sensitive fields such as passwords, card numbers, and health data, and give access by role. Privacy is not a later thought. It is part of the design.
If you rely on a cloud SIEM, you also have to know where the data is physically stored and what deletion commitments the contract carries.
Monitoring the Logging System Itself
The logging system has to monitor itself. Three things matter: source health, meaning whether every critical source really is forwarding; the parse failure rate, meaning whether a log format has changed; and queue depth, meaning whether the buffers are filling up. Without that monitoring you can believe everything is fine while a dead source or a full queue quietly loses data. A silent source is an invisible security hole.
Periodic Review, Not a One-Off
Central log architecture is not a one-off project. It is a continuous process. Correlation rules have to be reviewed, because stale rules generate dead alerts. New sources have to be added, retention has to be re-examined against new requirements, and capacity has to be re-examined against growth.
An organisation that buys a SIEM and then forgets about it ends up with a system full of false alerts that exhausts the team. Periodic review is part of the design, not an afterthought.
Key Decisions an Architect Has to Make
Alongside the ten principles above, an architect faces five key decisions, and each one determines the path of the project. None of them can be deferred. They have to be made at the start.
What Do We Collect?
Do not collect everything. Collecting everything explodes the cost and buries the signal in noise. Split the data into three classes instead.
- Critical data: authentication, access changes, and network connections. This is always collected.
- Operational data: errors and performance. Collected for troubleshooting.
- Debug data: usually dropped, or enabled only at the moment you need it.
That split controls the cost and raises the quality of the analysis at the same time.
How Long Do We Keep It?
Set retention against three criteria: the legal requirement, namely PCI-DSS, HIPAA, or SOC 2; the operational need, meaning how much history post-incident analysis actually requires; and cost. A common pattern is 30 days in Hot, 90 days in Warm, and 12 months in Frozen.
| Tier | Retention | What it is for |
|---|---|---|
| Hot | 30 days | Fast full-text search for current investigations |
| Warm | 90 days | Slower search for pattern and trend analysis |
| Frozen | 12 months | Cheap archive for compliance and rare deep searches |
This pattern satisfies the compliance obligation and keeps the bill under control at the same time.
Who Gets to See It?
Define access by role. A security analyst needs authentication and network logs, but not the logs of a financial application. A DevOps engineer needs application and performance logs, but not the security logs. A manager needs aggregate dashboards, not raw data. That separation raises security and privacy together.
What Should Be Automated?
Do not automate everything. Automation suits cases with a clear rule and a predefined response, such as locking an account after 5 failed logins. It is dangerous where human judgement is required, such as deciding whether something is an attack or a mistake. One wrong automated response can take a critical service down.
Which Tool for Which Problem?
Section 2 mapped the family of tools, and this decision follows from that map. Choose the tool by the problem you have, not by fashion or vendor pressure. A log management system is enough if you only need to collect and search.
A SIEM is needed for threat detection, and only if you have an analyst team to run it. Consider SOAR when incident volume exceeds what manual response can absorb. An XDR may suit you if you already run several security products and want one integrated view. The problem chooses the tool, not the other way round.
With a cloud SIEM the vendor is the processor and you remain the controller. Know where the data is stored, who can reach it, and what deletion commitments the contract carries at the end of the term.
Incremental Implementation Roadmap
Most organisations start from zero and have no idea where to begin. A gradual roadmap helps, and this one has five phases. Each phase delivers value on its own and can be run separately. The diagram below shows the order, with the foundation at the bottom and continuous maturity at the top.
Foundation, One to Two Months
The goal of this phase is infrastructure, not analysis. Three tasks matter.
First, put NTP in place. Every device synchronises to a single time source. This prerequisite governs everything that follows.
Second, stand up a central collector. A simple rsyslog or syslog-ng is enough. The goal is to get logs from the critical sources, the firewall, the main servers, and Active Directory, to one place.
Third, write the list of critical sources. Decide which devices must send logs and which must not. That list becomes the basis for source health monitoring in the later phases.
Searchability, Two to Four Months
The goal of this phase is to turn raw data into searchable data. Two tasks matter.
First, stand up a log management system. Elastic Stack, Loki, or Graylog. The goal is that an analyst can search the logs instead of reading files by hand.
Second, implement normalisation. Convert the different formats into one structure. Without it, search and correlation do not work.
At the end of this phase the organisation can answer the question, what happened?
Security Analysis, Four to Eight Months
The goal of this phase is to add the security analysis layer. Two tasks matter.
First, stand up a SIEM. Wazuh, Splunk, or Sentinel, depending on budget and need. The goal is to correlate security events.
Second, define the first use cases. Start with the classic rules: Brute Force, Impossible Travel, and Privilege Escalation. Those rules are the base of detection.
At the end of this phase the organisation can answer the question, is this dangerous?
Response and Automation, Eight to Twelve Months
The goal of this phase is to turn detection into response. Two tasks matter.
First, define the response procedure. When a high-priority alert fires, who receives it and what action follows? That procedure has to be written down and rehearsed.
Second, add SOAR if you need it. For simple, repeatable responses such as locking an account, automation can cut the response time.
At the end of this phase the organisation can answer the question, what should we do?
Write the response procedure before you automate anything. A playbook nobody has rehearsed is a guess executed at speed, and its first run will happen during a real incident.
Maturity and Review, Continuous
The goal of this phase is continuous improvement. Three tasks matter.
First, review the rules. Remove the stale ones, add the new ones, and reduce the false alerts.
Second, review the capacity. As the network grows, storage and buffer capacity have to be re-examined.
Third, review the compliance. Legal requirements change, and retention and access have to follow them.
Common Mistakes
Seven mistakes sink central logging projects more often than any technical limitation does. Each one is cheap to prevent at the start and expensive to repair at the end.
Buying the tool before defining the problem. Many organisations buy a SIEM because everyone else has one, without knowing which problem they want to solve. The result is an expensive tool that nobody uses.
Collecting everything. Collecting every log explodes the cost and buries the signal in noise. Filter on analytical value, not on the reasoning that it might be useful later.
Ignoring NTP. Clock drift destroys the order of events and makes correlation meaningless. NTP is a prerequisite, not an option.
Buffering without monitoring. A full queue discards data silently. Without monitoring you can believe everything is fine while data is being lost.
A SIEM without a team. A SIEM without trained analysts and tuned rules is just an expensive display. The tool does not replace the person.
Ignoring privacy. Logs contain personal data. Ignoring GDPR and similar laws carries legal risk and destroys trust.
Forgetting the review. Central log architecture is not a one-off project. Without periodic review the rules go stale, the alerts go false, and the system slowly becomes useless.
Do not buy the tool before you define the problem, and do not buy a SIEM without a team to run it. A SIEM that nobody tunes does not fail loudly; it fails quietly, and it burns analyst trust until somebody eventually switches the alerts off.
The Closing Argument
At the start of this document we said that large networks without central logging are blind. At the end we can say it more precisely: large networks without a well-designed central log system are blind, and a badly designed central log system is worse than blindness, because it gives an illusion of sight.
A central log system that only collects data but cannot be searched is a useless warehouse. A SIEM that produces alerts nobody handles is a useless watchman. An architecture with buffers nobody monitors is an illusion of security.
A good architecture starts with the ten principles above, makes the five key decisions, and matures across the five phases. The journey is not easy, but it is the only way an organisation can be certain that at the moment of an incident it knows what happened. That knowing is the difference between an organisation that was ready and one that was caught by surprise.
Summary
- Ten principles define the architecture: centrality, one clock, correlation from day one, layering, multi-layer buffering, a data lifecycle, least privilege, privacy by design, self-monitoring, and periodic review.
- Five decisions set the boundaries: what to collect, how long to keep it, who may see it, what to automate, and which tool fits which problem.
- Five phases deliver value in order, from a single time source and a central collector up to continuous maturity.
- Seven mistakes explain most failures: buying the tool first, collecting everything, ignoring NTP, unmonitored buffers, a SIEM without people, ignored privacy, and no review.
- A badly designed central log system is worse than no central log system at all, because it gives an illusion of sight.
Previous: Section 6 — Full Scenarios and Problem Solving · All sections