A Linux service runbook explains how to recognise a healthy service, investigate a failure and hand over the next action. It is more useful than a list of restart commands because it connects each action to evidence and a decision.
This guide is a documentation exercise for an authorised practice machine using systemd. The service is illustrative; it does not describe a client incident or provide a production recovery procedure. Use an existing lab where possible. If your lab uses a cloud machine, check its current charges and cleanup responsibilities before leaving it running.
Identify the service and its owner
Begin with the service's purpose, exact unit name, environment and responsible person. Explain who uses it and which dependencies it needs. A name such as “web server” is too vague if several services run on the machine. The next operator needs to know which component the runbook covers.
Record the Linux distribution and installed systemd version when you validate the document. Not every Linux environment uses systemd, and a container may be managed differently from its host. Link to the documentation appropriate to the actual environment. Never assume instructions tested on a personal lab apply unchanged to a shared server.
Define healthy before describing broken
Describe the normal observation from the user's point of view. For a practice web service, that might be a known page returning the expected content from the intended endpoint. Explain how the check is performed and what it does not establish. A homepage loading does not prove every dependent feature works.
Keep an example of the expected response using fictional data. Record the location of the application configuration and logs without copying credentials into the runbook. If another person needs access, say which authorised role can grant it. A handover document should make access understandable without becoming a secret store.
Separate runtime state from startup configuration
The systemd project's systemctl manual source distinguishes current runtime status from enablement. A status view reports runtime information and recent journal messages. Enabling a unit does not, by itself, start it. These are separate questions to record in a runbook.
For the practice service, document whether it should be running now and whether it should be activated at boot or by another mechanism. Do not change enablement simply because a service is stopped. It may be intentionally inactive or controlled by another part of the system. Ask the owner when the intended state is unclear.
Inspect before intervening
Write a short inspection sequence that preserves the evidence needed to understand the symptom. Start with the time, affected environment and reported behaviour. Then inspect the service state, relevant application messages and dependency observations using the access already authorised for the lab.
The systemd manual explains that status output focuses on the current or most recent invocation; journal queries can help inspect earlier invocations. Record the time window and service identity when collecting evidence. Remove secrets and personal information before sharing excerpts. A log line without its timestamp or surrounding context can point the next operator in the wrong direction.
- What symptom was reported, and when did it begin?
- Which environment and exact service are affected?
- Does the service state match the intended state?
- What do the relevant messages show around the failure?
- Was there a recent deployment or configuration change?
- Which dependency checks are relevant to this application?
Make starting a deliberate action
A start procedure should state its preconditions, required authority and expected effect. In your own lab, document the tested method for starting the specific service and the observation that confirms it became usable. Do not add a generic command copied from another application and call the procedure validated.
Distinguish starting a stopped service from restarting one that is serving users. Explain the possible interruption and what work could be affected. If a prerequisite is missing, record the blocker instead of repeatedly attempting the same action. This article deliberately leaves application-specific commands to your tested environment.
Write a bounded recovery decision
Choose a controlled practice scenario with a trainer or the lab owner. Describe the symptom, evidence to retain, permitted correction and rollback method before inducing any failure. Do not disrupt a shared machine to create a convincing demonstration. Keep the exercise limited to resources you are authorised to change.
A useful recovery entry says why an action is appropriate and when to stop. If the evidence suggests an invalid application setting, the correction might be restoring a previously validated configuration under the lab's procedure. It should not be disabling the firewall, bypassing access controls or weakening SELinux to make the symptom disappear.
Verify recovery and record what remains unknown
After an authorised correction, repeat the original user-facing check and the relevant dependency checks. Compare the observed result with the expected result. A successful command exit alone is not evidence that the application now performs its intended task.
Record any remaining error, incomplete check or temporary workaround. If the symptom returns, stop repeating the same recovery and escalate with the evidence already collected. The runbook should help someone recognise the limit of their procedure, not encourage unlimited retries.
Leave a useful handover
Close the record with the current state, actions performed, checks completed, unresolved issues and next owner. Include the validation date and where the supporting evidence is stored. Keep historical incident notes separate from the current procedure so an old workaround does not silently become the default instruction.
Ask another authorised learner to read the document and identify missing assumptions. Can they find the intended service, understand the health check and tell when approval is required? Revise unclear steps, then record which parts were actually exercised and which remain untested.
Common questions
Is an active service necessarily healthy?
No. Runtime state is one observation. Check the application behaviour that matters to its users and the relevant dependencies before declaring recovery.
Should I restart first?
Follow the approved procedure for the environment. In this learning exercise, collect relevant evidence first and establish whether a restart is authorised and appropriate.
Can I reuse the runbook on another server?
Reuse the structure, then validate the service name, platform, access, dependencies and recovery steps for that environment. Mark untested instructions clearly.
Explore Linux training or bring your practice runbook to free counselling to discuss an appropriate learning path.