Tired of hunting through logs? Here’s how I automated the entire process


If you’ve ever debugged a web application, you’ve probably done this more than you’d like to admit. An error pops up. You scroll through your logs. You find what looks important. You copy a chunk of it. Then you paste it somewhere. Maybe into a chat with a teammate, maybe into a document, or increasingly, into an AI tool to help you make sense of it.

I’ve been in your shoes, and at some point, I felt like this needed to stop. So, I carefully crafted a solution that takes care of this manual log handling.

Crafting an automation system for parsing logs

Letting a script handle manual workflow

Home Assistant C.A.F.E. automation flowchart editor integration. Credit: Tim Brookes / How-To Geek

Logs are already structured text. With a small script, I could pull out only the important parts, clean them up, and format them into something readable, automatically. The idea was simple: take the exact workflow I was repeating by hand, and turn it into something I could run with a single command. A reusable script.

To test this idea, I used a tiny demo web app that randomly throws errors and writes them to a log file. If you want to follow along, I’ve put the code up on my GitHub. But you don’t need this exact setup. Any application that writes logs to a file will work the same way. The only thing that matters for now is a growing log file with a mix of useful errors and some noise around them.

Instead of manually digging through logs every time, I wrote a small, smart Python script to do the boring part for me. The idea is simple: read the log file, pull out only the error sections, clean them up, and save them into a Markdown file I can use anywhere. Here’s a minimal version of the script:

from datetime import datetime

LOG_FILE = "app.log"


def is_new_log_entry(line):
    return any(level in line for level in ["INFO", "WARNING", "DEBUG", "ERROR"])


def extract_errors(lines):
    errors = []
    current_error = []
    capturing = False

    for line in lines:
        if "ERROR" in line:
            if current_error:
                errors.append("".join(current_error))
                current_error = []
            capturing = True

        elif capturing and is_new_log_entry(line):
            errors.append("".join(current_error))
            current_error = []
            capturing = False

        if capturing:
            current_error.append(line)

    if current_error:
        errors.append("".join(current_error))

    return errors


def format_markdown(errors):
    formatted = []

    for i, error in enumerate(errors, 1):
        block = f"### Error {i}\n```\n{error.strip()}\n```"
        formatted.append(block)

    return "\n".join(formatted)


def save_report(content):
    timestamp = datetime.now().strftime("%Y-%m-%d_%H-%M")
    filename = f"error_report_{timestamp}.md"

    with open(filename, "w") as f:
        f.write("# Error Report\n")
        f.write(content)

    print(f"Saved: {filename}")


def main():
    with open(LOG_FILE, "r") as f:
        lines = f.readlines()

    errors = extract_errors(lines)

    if not errors:
        print("No errors found.")
        return

    report = format_markdown(errors)
    save_report(report)


if __name__ == "__main__":
    main()

There are four small steps happening here:

  1. Read the log file: The script loads the log file into memory as a list of lines. This makes it easier to process logs one step at a time instead of dealing with one big chunk of text.
  2. Extract only error logs: Whenever a line contains “ERROR”, the script treats that as the beginning of a new error block. From that point on, it starts collecting every line that belongs to that error, including the full traceback.
  3. Capture the full error block: Instead of guessing where the error ends, the script keeps collecting lines until it hits the start of a new log entry (like “INFO” or another “ERROR”.) This ensures you get the entire stack trace without cutting anything off.
  4. Format the output: Each error is wrapped in a clean Markdown block, making it easy to read, share, or paste into other tools.
  5. Save it with a timestamp: Finally, the script writes everything to a new file with a timestamped name, so you don’t overwrite previous reports:
error_report_2026-04-16_14-32.md
3D logos of Python and Microsoft Excel side by side on a blue and yellow background.


Python in Excel isn’t just for programmers—4 useful things you can do with it right now

Turn Excel into a lightweight data-science tool for cleaning datasets, standardizing dates, visualizing clusters, and analyzing keywords.

The result

What my workflow looks like with the script in action

After hitting the error endpoint a few times, my raw log file (app.log) looked like this:

2026-04-17 19:57:58,880 - INFO - Root endpoint was called
2026-04-17 19:58:17,147 - INFO - Root endpoint was called
2026-04-17 19:58:32,806 - INFO - Root endpoint was called
2026-04-17 19:58:42,878 - INFO - Error endpoint was called
2026-04-17 19:58:42,878 - ERROR - Exception occurred in /error endpoint
Traceback (most recent call last):
  File ".../main.py", line 28, in trigger_error
    result = 10 / 0
ZeroDivisionError: division by zero
2026-04-17 19:58:56,878 - INFO - Root endpoint was called
2026-04-17 19:58:59,833 - INFO - Root endpoint was called
2026-04-17 19:59:02,997 - INFO - Error endpoint was called
2026-04-17 19:59:02,997 - ERROR - Exception occurred in /error endpoint
Traceback (most recent call last):
  File ".../main.py", line 28, in trigger_error
    result = 10 / 0
ZeroDivisionError: division by zero
...
A log file showing different types of logs from a web application.

It’s a mix of everything: successful requests, repeated noise, and the actual errors buried in between. After running the script, I got a clean Markdown report:

# Error Report

### Error 1

```
2026-04-17 19:58:42,878 - ERROR - Exception occurred in /error endpoint
Traceback (most recent call last):
  File "D:\Content Writing\How-To Geek\automate logs\demo-webapp-error\main.py", line 28, in trigger_error
    result = 10 / 0
             ~~~^~~
ZeroDivisionError: division by zero
```
...
A clean Markdown file containing structured error outputs captured and cleaned from a log file.

Each error is isolated, complete, and already formatted in a way that’s easy to read or share.

Now, you can use this script and take the automation to the next level by integrating Slack, an LLM, or any other service where you’d want to copy and paste this log. That way, the logs will automatically reach where you want them to be.

Hands typing on a white keyboard, surrounded by floating Bash scripting code snippets and icons against a green background.


18 Bash string tricks that fix common scripting headaches

Unlock the power of Bash strings with these clever techniques.


Automation saves the day

This was a fun experiment, but it has changed my outlook towards scripting and automation. With such a simple but useful Python script, I was able to save hours of headache and manual work.

apple mac mini 2024 with m4 pro

Brand

Apple

Operating System

macOS




Source link

Leave a Reply

Subscribe to Our Newsletter

Get our latest articles delivered straight to your inbox. No spam, we promise.

Recent Reviews


Serials have become the backbone of the streaming era, especially on Netflix. Serialized television is when a show’s plot unfolds in sequential order over the course of a season. It’s long-form storytelling that typically works best with dramas—Stranger Things, The Crown, etc. Watching the episodes in release order matters. Often, these shows are binged because the complex character arcs and cliffhangers encourage streaming multiple episodes at once.

Serial shows can feel like homework, especially when you fall behind on an episode and need to catch up. That always happens to me, and it leads to anxiety I didn’t want. Thankfully, Netflix offers shows where viewers can jump at any time and not feel lost. These episodic series are perfect for jumping around and picking the episodes you want to watch. One of the most famous comedies ever fits the criteria of an episodic sitcom. Anthology shows, including a Netflix sci-fi classic, are also ideal for watching episodes out of order.

Black Mirror

Welcome to your worst nightmare

Black Mirror wants to scare you. Charlie Brooker’s sci-fi anthology series has been warning humanity about the dangers of technology since 2011. It seems like ages ago that Rory Kinnear had sexual intercourse with a pig in the first episode. Apologies for the spoiler, but the media’s role in the spread of misinformation has never been more relevant.

Black Mirror features self-contained episodes with a beginning, middle, and an end. There has only been one direct sequel: USS Callister: Into Infinity, a season 7 episode that continues the events of season 4’s USS Callister. Otherwise, feel free to jump around and check out the best episodes of each season. Since most episodes feature bleak endings, I’ll leave you with one that ends on an upbeat note: San Junipero.

Seinfeld

Greatest comedy ever?

Comedies are the perfect vehicle for episodic storytelling. While having an overarching plot throughout a season helps attract viewers, many comedy fans are just looking for a few laughs. Write a self-contained story with numerous jokes over 20 to 30 minutes, and you’re ready to go. Seinfeld, aka the show about nothing, is the ideal escape from serialized dramas.

Seinfeld stars Jerry Seinfeld as a fictionalized version of himself as he navigates the comedic scene in New York City. The show revolves around Jerry’s interactions with his friends George (Jason Alexander), Elaine (Julia Louis-Dreyfus), and Kramer (Michael Richards). The gang faces a problem, hilarity ensues, and the episode ends. That’s really all you need to know. Enjoy the laughs.

Guillermo del Toro’s Cabinet of Curiosities

The genre maestro curates new horror stories

There’s a reason why Guillermo del Toro is considered the “King of the Monsters.” The genre expert is as elite as it comes when dealing with mythology and creating new worlds. The Oscar winner relied on his horror expertise in the anthology series Guillermo del Toro’s Cabinet of Curiosities.

I hate referring to episodes of television as “mini-movies.” However, that’s how I would describe the eight episodes of Cabinet of Curiosities. Each director puts their own signature style on a story and brings audiences into their terrifying creation. Del Toro wrote two of the episodes, including one about a demon being summoned. Some are scarier than others, but horror fans will feel right at home with this series. ​​​​​​​

Beat Bobby Flay

Bobby brings the heat

As I’ve gotten older, the Food Network has become one of my favorite channels. I mean, who doesn’t love food? I love eating my (average) home-cooked meal while watching contestants duke it out in the kitchen on my favorite show, Beat Bobby Flay. The competition breaks down into two rounds. In the first round, two chefs have 20 minutes to construct a meal using a secret ingredient. The winner advances to the main event, where they face off against Bobby Flay.

The challenger gets to pick the dish for the final round, so Bobby has a disadvantage. However, Bobby is an award-winning chef with a few tricks up his sleeves. He can handle making a version of your grandmother’s lasagna. With episodes available on Netflix, be prepared to learn why Bobby always throws chiles into his dishes.​​​​​​​

S.W.A.T.

Broadcast TV still knows how to make entertaining programs

The procedural is a genre best produced on broadcast television. Name a cop, doctor, or law drama—chances are it’s a procedural on broadcast TV. While the way we watch television has changed, people still love these types of shows on CBS, NBC, Fox, and ABC. Law & Order, NCIS, and Criminal Minds are procedurals that gained a bigger following thanks to streaming.

S.W.A.T. is cut from the same cloth as Chicago P.D. and CSI. Sergeant Daniel “Hondo” Harrelson (Shemar Moore) is tasked with leading a new S.W.A.T. unit in the LAPD. This action-packed show utilizes a “case of the week” formula in which the team must solve a dangerous situation, such as active shooters and hostage situations. You’re in and out in 44 minutes. What’s better than that?​​​​​​​


Netflix has more content coming your way

After you’re done watching these shows, stay on Netflix for more top-notch content. Netflix has an entire section dedicated to thrillers, and this week, The Guilty and El Camino are two of the section’s best. Keep an eye out for new movies, like Alan Ritchson’s War Machine, which is currently in the streamer’s top 10.

Subscription with ads

Yes, $8/month

Simultaneous streams

Two or four




Source link