Manual Web App Security Checks Don't Scale: Inside Our Automated Assessment & Remediation Framework

Security Engineering · Python · OWASP

Imagine a vulnerability report featuring 40 issues but lacking a definite sequence; effectively, it's just noise. We are sharing our experience in creating a tool that not only gives you the exact order of fixing but also displays the grade you'll achieve after doing it.

Picture this: A security scanner completes its work scanning your web application, for instance. It returns to you a list of 40 findings, a combination of CRITICAL, HIGH, MEDIUM, and LOW severity. Without any order. Without any context. Without any indication of which five must be fixed by the end of the day, the remaining twenty can wait until the next sprint.

This is the situation for many small engineering teams working on security. The tools are not defective, they are simply lacking in some features. They inform you of what the problems are but not how to solve them, or which ones to tackle first, or what the improvements will be after you've fixed them.

01 — Why Checklists Break Down at Scale

If you've ever deployed a web application, you're managing at least four distinct security surfaces simultaneously, whether you realise it or not:

If a single layer is misconfigured, the rest will be completely compromised. If the container itself is compromised, it does not matter that the hardened Flask app running in a Docker container with --privileged has not been compromised. Manual checklists treat these layers separately. Many automated tools do the same.

The two gaps that matter most in practice are:

• No prioritisation signal. When everything is labelled HIGH, nothing is. Teams freeze or default to tackling whatever looks familiar.
• No forward visibility. You can't see how your security posture changes as you apply fixes. Remediation feels like work with no feedback loop.

02 — What the Framework Does

The framework performs 24+ opinionated checks across all four layers, maps each finding to the OWASP Top 10:2025, then produces three things a standard scanner won't:

• A priority score for every failing check, based on severity, effort to fix, and estimated risk reduction
• A 30-day hardening roadmap grouped into Day 1, Day 7, and Day 30 phases
• A what-if simulation showing your projected grade and attack path exposure after each phase

It's implemented as a modular Python CLI. Each check returns a simple CheckResult object:

@dataclass
class CheckResult:
    id: str            # e.g. "APP-DEBUG-001"
    layer: str         # app | webserver | container | host
    name: str
    status: Status     # PASS | FAIL | WARN | ERROR
    severity: Severity # CRITICAL | HIGH | MEDIUM | LOW
    details: str

Clean, flat, composable. A full scan aggregates these into a ScanResult that computes your overall pass rate, letter grade, OWASP category breakdown, and detected attack paths.

03 — Prioritisation: Effort vs. Impact

Here is the essential point that distinguishes this from just a checklist: only some HIGH-severity issues are really worth fixing first. For example, a HIGH problem that can be solved with a one-line configuration change should be prioritized ahead of a HIGH problem that requires an architectural refactor, even if both have the same risk level.

This is modeled in the system as extending all the check definitions with two new separate fields: effort (one of LOW / MEDIUM / HIGH) and impact_weight (a float that modulates risk reduction).

Combined they provide the following simple priority formula:

priority = (severity_score × impact_weight) ÷ effort_score

High severity + high impact + low effort = fix this today.

For example, disabling Flask debug mode in production is a single configuration flag (DEBUG=False), carries a High severity, and significantly reduces information leakage risk. It scores 4.0 — a Day 1 item. Contrast that with a MEDIUM finding requiring a container rebuild: same severity bucket, lower priority score, later phase.

Here's what the check metadata looks like in practice:

{
    "id": "APP-DEBUG-001",
    "layer": "app",
    "name": "Debug mode disabled",
    "severity": "HIGH",
    "owasp": ["A02:2025"],
    "effort": "LOW",         # one config flag
    "impact_weight": 2.0,    # high risk reduction if fixed
    "recommendation": "Set DEBUG=False in Flask/Django settings.",
}

04 — The 30-Day Roadmap: From F to A

Once checks are scored, the framework sorts them and buckets them into a 30/40/30 split — the top 30% of priority items become Day 1 work, the next 40% become Day 7 work, and the remainder form a Day 30 backlog.

But the most useful feature isn't the plan itself — it's the simulation that runs alongside it. The framework temporarily marks each phase's fixes as PASS and recomputes your grade, score, and attack-path count. Here's what a real output looks like:

Notice the inflection point at Day 7: that's when the last active attack path closes. This kind of forward visibility changes how teams talk about security work. Instead of "we need to fix 24 things", the conversation becomes "seven fixes this week eliminates our only active attack path".

What's an attack path? An attack path is a chain of related misconfigurations across layers that, taken together, could allow an attacker to escalate access. For example: debug mode enabled (leaks internal routes) + weak session cookies (no HttpOnly flag) + no rate limiting = a credible path to account takeover. The framework detects these cross-layer chains and counts them separately from individual findings.

05 — Running Your Own What-If Scenarios

Beyond the automatic roadmap simulation, you can test arbitrary hypotheticals directly from the CLI. Say you're deciding which two fixes to prioritise this sprint:

python audit.py --simulate HOST-FW-001,APP-COOKIE-001

Under the hood, the simulator constructs a temporary copy of your scan result, marks the specified checks as PASS, and recomputes all metrics fresh. Nothing is persisted — it's purely a projection:

def simulate_with_fixes(self, fix_ids: list[str]) -> dict:
    simulated_checks = []
    for c in self.checks:
        if c.id in fix_ids and c.status != Status.PASS:
            # Treat this check as fixed
            simulated_checks.append(
                CheckResult(..., status=Status.PASS)
            )
        else:
            simulated_checks.append(c)
 
    sim_result = ScanResult(checks=simulated_checks)
    return {
        "simulated_grade": sim_result.grade.value,
        "simulated_score": sim_result.score_percentage,
        "attack_paths": sim_result.attack_path_count,
    }

The elegance here is that there's no special simulation mode — the same scoring and grading logic runs on both real and simulated data. If the scoring model is accurate, the projection is too.

06 — Same Data, Different Audiences

One thing security reports almost never do is adapt to their reader. A student learning about web security needs different language than a CTO deciding sprint priorities. The framework addresses this with a --profile flag that adjusts the narrative framing of OWASP findings without changing the underlying data.

Here's the same finding — debug mode enabled — described for three different readers:

👩‍🎓 Student

"Debug mode left on in production is one of the most common beginner mistakes — it exposes stack traces that reveal your app's internal structure to anyone who triggers an error."

⚙️ DevOps

"DEBUG=True must never reach production. Add this to your deployment pipeline as a gating check. Pair with a secrets scanner in CI to catch config drift early."

📊 CTO

"Three HIGH findings in the application layer represent material risk. The Day 1 remediation sprint addresses all three with low engineering effort and closes the active attack path."

This isn't cosmetic. For educational contexts especially, the ability to reframe the same OWASP finding as a learning moment versus a business risk versus an ops task makes the tool genuinely useful across different settings.

07 — What's Actually New Here

To be clear: this framework doesn't invent new security checks. OWASP's guidance has been well-established for years. What it contributes is different:

• Cross-layer attack path detection — linking related findings across app, server, container, and host into coherent exploit chains
• Cost-aware prioritisation — the severity × impact ÷ effort heuristic produces actionable ordering, not just a risk label
• Forward simulation — quantifying how partial remediation improves posture before any work is done
• Audience-adaptive reporting — the same findings, framed differently for learners, engineers, and decision-makers

The combination of these four things is what makes it useful for small teams. A two-person startup doesn't need a 40-item audit report. They need to know: what are the seven things we fix this week to close our attack paths?

Security tooling should be decision support — not just documentation of what's wrong.

08 — Where This Goes Next

There are natural extensions worth exploring. The effort and impact weights are currently static — a learning model trained on remediation outcomes could make them adaptive over time. The attack-path detection is heuristic-based; a graph model of the four layers could make it more precise. And the profile system could be extended to generate role-specific ticket templates directly into project management tools.

But even in its current form, the core loop — scan → prioritise → simulate → report — addresses the two gaps that matter most in practice. It turns a static findings list into a time-bounded plan, and it gives teams a way to talk about security investment in terms of outcomes rather than effort.

Try it yourself: The total source along with more than 24 check descriptions, the PDF creation tool, and command line interface simulation flags can be found at GitHub or better still search the GitHub repo: Security Audit Framework. It is an open source framework which can be run against any stack of Flask/Nginx/Docker/Linux where it won't require any agent simply direct it towards a host.

I really appreciate it if you've come this far thank you for reading.

Really, I would love to get your feedback. Have you encountered the same priority-setting issue with your team? Is there a layer or check category missing, in your opinion? Or perhaps you would score the effort/impact the other way? Share your thoughts in the comments below, pass it on to that person with the 40-item findings list, and lastly, if you like the framework, please give it a ⭐ on GitHub it helps to get it to more people.