Validate Test Coverage Report
quality.validate_test_coverage_report
Overview
You send a coverage report from JaCoCo, Cobertura, Clover, LCOV, or Istanbul, inline or as a file you uploaded, and we work out the format, read the covered and total counters out of it, and recompute every percentage ourselves rather than trusting the numbers printed in the file. Then we hold it to your rules — an overall bar, a bar every file has to clear, or a stricter bar for the files a change touched — and hand back one entry per rule that was missed. Tests are never run and nothing is fetched.
Every coverage tool writes a different file and rounds its percentages its own way, so a bar that means one thing in one project quietly means another elsewhere.
Reliability
The same report and the same bars always give the same verdict, and nothing leaves the platform — XML is read with document type definitions and external entities disabled, so a hostile report cannot reach anything. Percentages are always recomputed from the covered and total counters and rounded to two decimals, so a report that rounds generously cannot talk its way past your bar, and a metric with a total of zero comes back as null rather than 0 or 100. Paths are normalised after your rewrites: backslashes become slashes, a leading slash or "./" is dropped, and anything still trying to climb above the project is refused. A missed bar is a normal successful answer — read valid and violations. Four failures stop the call. COVERAGE_REPORT_MALFORMED means the file does not parse as the format it claims, and COVERAGE_FORMAT_UNKNOWN means we could not tell what it is; for both, check what actually produced the file and name the format explicitly, then retry. COVERAGE_REPORT_EMPTY means the content is blank, or it parsed to nothing at all while failOnEmptyReport is on, which usually means the test run failed before writing anything. INVALID_PATH_MAPPING means a path, after your rewrites, points outside the project, so correct the from and to values. Retrying the last two unchanged will fail identically.
Context footprint
The reply holds the totals and one line per missed bar, never the report itself, so it stays small unless thousands of files each miss — lower maxViolations when that is a risk. A successful call costs 1 execution credit.
When to use it
When a coverage report exists and something has to decide whether it is good enough.
When the report still has to be produced, or you want to browse the results.
Runs on the JDK's XML parser, hardened against external entities, and Jackson.
Scope
Non-goals
- Does not run your tests or produce the report
- Does not fetch a report from a URL or a build server
- Does not render coverage as HTML or any other view
- Does not tell you which lines are uncovered — only the counts per file
Properties
Capabilities
Cost & billing
Flat rate per invocation
File handling
File limits
Constraints this skill applies to your input and generated files. The “Limited by” column shows the static source of each cap; none of these can be raised by upgrading your plan.
| Limit | Value | Limited by |
|---|---|---|
| Maximum size per input file | 20 MiB | Skill and platform maximum |
How we handle your data
What this means
This skill may receive sensitive content, but it does not target sensitive data and returns only a structured non-sensitive result.
Squillify guarantees
- Runs inside Squillify
- This skill runs inside Squillify.
- Payloads are not sent to an external service by this skill.
- Squillify does not permanently store submitted payloads.
- Telemetry records metadata only, not submitted content.
- Submitted payloads are not used for training.
- Submitted payloads are not reviewed by humans.
- This skill does not target sensitive data.
- The response does not contain submitted content.
Your responsibility
Squillify does not determine whether submitted content is sensitive. Your application is responsible for deciding what data it is allowed to submit. Your application is responsible for securing the response.
Input schema
| Field | Type | Required | Description |
|---|---|---|---|
| source | object | Yes | Closed report source: type inline with content, or type file_ref with a tenant-owned ref. |
| format | string | No | Report format to parse; omit it for auto detection from the content. |
| thresholds | object | No | Closed optional bars: global totals, per-file bars and changed-file bars; omit a part to skip that check, or omit the object to report metrics only. |
| pathMapping | array | No | Prefix rewrites, 0 through 256 closed {from, to} objects; omit it to keep report paths. |
| exclusions | array | No | Exclusion globs, 0 through 256 strings; omit it to check every file. |
| options | object | No | Closed empty-report, missing-metric and violation-cap controls; omit the object for fail-closed defaults and 1000 violations. |
Output schema
| Field | Type | Required | Description |
|---|---|---|---|
| valid | boolean | Yes | True when every supplied bar was cleared. |
| format | string | Yes | Format the counters were actually read as after detection or an explicit choice. |
| metrics | object | Yes | Closed report totals keyed by metric name; each value is covered, total and percentage. |
| violations | array | Yes | Missed bars, 0 through 10000 closed objects; empty when everything cleared. |
| warnings | array | Yes | Non-fatal advisory notices as 0 through 100 strings, each one sentence long; empty when nothing was worth flagging. |
| _meta | object | Yes | Closed read counts with entriesRead from the report and excludedEntries removed by globs. |
API contract
POST /v1/executecurl -sS -X POST "$BASE/v1/execute" -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"skillKey":"quality.validate_test_coverage_report","input":{"source":{"type":"inline","content":"TN:\nSF:a.js\nLF:10\nLH:5\nend_of_record\n"},"format":"lcov","thresholds":{"global":{"lines":80}}}}'import { Squillify } from "@squillify/sdk";
const sq = new Squillify({ apiKey: process.env.SQUILLIFY_API_KEY });
const { output } = await sq.execute("quality.validate_test_coverage_report", {"format":"lcov","source":{"type":"inline","content":"TN:\nSF:src/a.js\nLF:10\nLH:5\nBRF:4\nBRH:2\nFNF:2\nFNH:2\nend_of_record\n"},"thresholds":{"global":{"lines":80}}});Errors
| Code | HTTP | Retryable | Description |
|---|---|---|---|
| VALIDATION_ERROR | 400 | No | The request payload failed schema validation. |
| UNAUTHORIZED | 401 | No | Missing or invalid API key. |
| SKILL_NOT_ACCESSIBLE | 403 | No | The API key is not granted access to this skill. |
| SKILL_NOT_AVAILABLE_FOR_PLAN | 403 | No | This skill is not included in your current plan. |
| PLAN_ELIGIBILITY_UNAVAILABLE | 503 | Yes | Plan eligibility could not be decided. Retry after backoff. |
| SKILL_NOT_FOUND | 404 | No | The requested skill or version does not exist. |
| TENANT_RATE_LIMITED | 429 | Yes | Request rate limit exceeded. Retry after backoff. |
| QUOTA_EXCEEDED | 429 | Yes | Plan quota exhausted. Retryable once the quota period resets. |
| EXECUTION_TIMEOUT | 504 | Yes | Execution exceeded the allowed time budget. |
| COVERAGE_REPORT_MALFORMED | 400 | No | The request payload failed this skill's input validation. |
| COVERAGE_FORMAT_UNKNOWN | 400 | No | The request payload failed this skill's input validation. |
| COVERAGE_REPORT_EMPTY | 400 | No | The request payload failed this skill's input validation. |
| INVALID_PATH_MAPPING | 400 | No | The request payload failed this skill's input validation. |
Other limits
| Limit | Value |
|---|---|
| Recommended inline bytes | 32 KiB |
Limitations
- Does not run your tests or produce the report
- Does not fetch a report from a URL or a build server
- Does not render coverage as HTML or any other view
- Does not tell you which lines are uncovered — only the counts per file
- Running tests or generating the report
- Producing a browsable coverage view
How to call
POST /v1/execute (requires authentication)
{
"skillKey": "quality.validate_test_coverage_report",
"input": {
"format": "lcov",
"source": {
"type": "inline",
"content": "TN:\nSF:src/a.js\nLF:10\nLH:5\nBRF:4\nBRH:2\nFNF:2\nFNH:2\nend_of_record\n"
},
"thresholds": {
"global": {
"lines": 80
}
}
}
}Examples
Half the lines covered, against a bar of 80
{
"format": "lcov",
"source": {
"type": "inline",
"content": "TN:\nSF:src/a.js\nLF:10\nLH:5\nBRF:4\nBRH:2\nFNF:2\nFNH:2\nend_of_record\n"
},
"thresholds": {
"global": {
"lines": 80
}
}
}{
"metrics": {
"functions": {
"covered": 2,
"percentage": 100,
"total": 2
},
"lines": {
"covered": 5,
"percentage": 50,
"total": 10
},
"branches": {
"covered": 2,
"percentage": 50,
"total": 4
}
},
"_meta": {
"entriesRead": 1,
"excludedEntries": 0
},
"violations": [
{
"code": "GLOBAL_THRESHOLD_NOT_MET",
"metric": "lines",
"actual": 50,
"required": 80,
"path": null
}
],
"valid": false,
"format": "lcov",
"warnings": []
}Fully covered lines, with metrics the report never reported
{
"format": "lcov",
"source": {
"type": "inline",
"content": "TN:\nSF:src/a.js\nLF:10\nLH:10\nend_of_record\n"
},
"thresholds": {
"global": {
"lines": 80
}
}
}{
"metrics": {
"functions": {
"covered": 0,
"total": 0,
"percentage": null
},
"lines": {
"covered": 10,
"percentage": 100,
"total": 10
},
"branches": {
"covered": 0,
"total": 0,
"percentage": null
}
},
"_meta": {
"entriesRead": 1,
"excludedEntries": 0
},
"violations": [],
"valid": true,
"format": "lcov",
"warnings": []
}Recommended use cases
- Stop a merge when coverage drops below the bar you agreed on
- Compare coverage across projects that use different report formats
- Hold the files a change touched to a higher bar than the rest
Avoid for
- Running tests or generating the report
- Producing a browsable coverage view
Related skills
Composable with
Safety & privacy
- Runs without calling any external service.
- Your payloads are never stored permanently.
- Telemetry captures metadata only — never your payload contents.
- We never train models on your payloads.
- No human reviews your payloads.