Running tests from the CLI: how dotnet test discovers and runs tests, VSTest filters, loggers and TRX results, Coverlet and Microsoft code coverage, hang and crash diagnostics, parallelism, and Microsoft.Testing.Platform in .NET 10.
dotnet test builds the test projects and runs them through the test platform, using the adapter the project references (xUnit, NUnit or MSTest). I use --filter to pick tests, for example --filter "Category=Unit&FullyQualifiedName~Orders". I use --logger trx so CI can show the results, and --collect "XPlat Code Coverage" with Coverlet for coverage, which I turn into a report. When tests hang or crash, --blame-hang-timeout and --blame-crash capture dumps. The command exits non-zero on failure, so the pipeline fails. In .NET 10 the SDK can also run in Microsoft.Testing.Platform mode, which you opt into in global.json.
"
dotnet testbuilds the test projects, unless I pass--no-buildafter one solution build, and runs them through the test platform using the adapter each project references, xUnit, NUnit or MSTest, alongsideMicrosoft.NET.Test.Sdk. Test projects run in parallel by default, and within a project the framework's own rules decide. The exit code is non-zero on any failure, so the pipeline must never swallow it.I use
--filterto pick tests byFullyQualifiedName, name or category, with operators for equals, contains, and, and or, which lets me split fast unit tests from slower integration tests into separate stages.--logger trxwrites results CI can display, or JUnit through a logger package, and when I wire a pipeline I publish them even when tests fail. For coverage I add--collect "XPlat Code Coverage"withcoverlet.collector, or Microsoft's collector, and treat it as a signal for untested risk, not a target.When a run hangs or crashes,
--blame-hang-timeoutand--blame-crashcapture a dump, so a mysterious timed-out pipeline becomes something I can analyse. In .NET 10 the SDK can also run in Microsoft.Testing.Platform mode, opted into inglobal.json; its options differ, so the mistake I see is switching modes without updating the pipeline arguments and silently losing results and coverage. I keep run settings and filters in the repo, so anyone can reproduce exactly what CI ran."
dotnet test on a solution or project builds the test projects (skip with --no-build).xunit.runner.visualstudio, NUnit3TestAdapter, MSTest.TestAdapter) referenced by the project, together with Microsoft.NET.Test.Sdk.Test projects run in parallel by default; within a project, the framework decides (xUnit runs test collections in parallel, for example).
--filter "FullyQualifiedName~Orders.Domain" # contains
--filter "Category=Integration" # NUnit/MSTest category, xUnit [Trait("Category", ...)]
--filter "Category!=Slow&FullyQualifiedName~Pricing" # and
--filter "Name=Total_includes_gst|Name=Total_rounds" # or
Use filters to split fast unit tests from slower integration tests into separate pipeline stages once the full suite takes too long for every push.
--logger trx --results-directory ./TestResults writes TRX files that Azure DevOps and most CI systems display; logger packages add JUnit output for others.--collect "XPlat Code Coverage" uses coverlet.collector (included in the xUnit template) to write Cobertura XML; ReportGenerator turns it into HTML and summaries. Microsoft's own collector (--collect "Code Coverage") is another option.--blame-hang-timeout 5m collects a dump and kills the test host if a test runs longer than the timeout; --blame-crash collects a dump if the host crashes. Both turn a mysterious timed-out pipeline into something you can analyse.
Microsoft.Testing.Platform (MTP) is the newer, lighter test platform: test projects built on it are executables you can run directly, and MSTest, NUnit, xUnit v3 and TUnit all support it. Historically dotnet test drove MTP projects through a VSTest bridge. The .NET 10 SDK adds a native MTP mode for dotnet test, which you opt into per repository in global.json:
{
"sdk": { "version": "10.0.100" },
"test": { "runner": "Microsoft.Testing.Platform" }
}
In that mode, options follow MTP's syntax (for example its own report and coverage extensions) rather than VSTest's --logger and --collect. VSTest remains the default and is fully supported, so switch a large solution deliberately: confirm every test framework in it supports MTP, update pipeline arguments, and check that your CI still picks up results.
A pipeline intermittently timed out after 60 minutes with no clue which test hung. Adding --blame-hang-timeout 10m produced a dump on the next occurrence, which showed an integration test waiting on a TaskCompletionSource that a mocked message bus never completed. The fix took ten minutes once the evidence existed.
The interviewer is checking practical CI testing skills, not just writing tests.
Green flags
dotnet test finds and runs tests via adapters.Red flags
dotnet test Orders.slnx -c Release
|
+-- build each test project (unless --no-build)
|
v
test platform (vstest.console)
|
+-- for EACH test project and EACH target framework:
| start a separate testhost process
| loads Microsoft.NET.Test.Sdk + the adapter package
| (xunit.runner.visualstudio / NUnit3TestAdapter / MSTest.TestAdapter)
| adapter discovers tests by reflection, then runs them
|
+-- data collectors attach to the testhost
| coverage (coverlet / Microsoft Code Coverage), blame (dumps)
|
+-- loggers receive results
| console, trx, junit (via a logger package), html
|
v
exit code: 0 = all passed, non-zero = a failure or a run error
The separate testhost process isolates tests from the runner. It is how collectors attach, how a crash in a test doesn't take down the runner, and how blame can dump a hung host.
--filter "<property><operator><value>", combined with & (and), | (or) and parentheses.
| Operator | Meaning |
|---|---|
= |
Exact match |
!= |
Not equal |
~ |
Contains |
!~ |
Does not contain |
| Framework | Supported properties |
|---|---|
| xUnit | FullyQualifiedName, DisplayName, and any trait name, such as Category from [Trait("Category", "Unit")] |
| NUnit | FullyQualifiedName, Name, Priority, TestCategory / Category |
| MSTest | FullyQualifiedName, Name, ClassName, Priority, TestCategory |
Examples:
--filter "Category=Unit"
--filter "FullyQualifiedName~Orders.Domain"
--filter "Category!=Slow&FullyQualifiedName~Pricing"
--filter "(Category=Unit|Category=Contract)&FullyQualifiedName!~Legacy"
Characters such as (, ), & and | inside a value must be escaped with \, and in a shell the whole filter must be quoted.
| Logger | How | Used by |
|---|---|---|
| Console | Built in; --logger "console;verbosity=detailed" for more output |
Local runs, CI logs |
| TRX | --logger trx --results-directory ./TestResults |
Azure DevOps PublishTestResults@2, most CI reporters |
| JUnit | --logger junit with the JunitXml.TestLogger package |
GitHub Actions reporters, GitLab, Jenkins |
| HTML | --logger html |
Quick human-readable output |
Publish the result files as a pipeline step that runs even when tests fail (condition: always() in Azure Pipelines, if: always() in GitHub Actions), otherwise the failures you most need to see are never published.
dotnet test --collect "XPlat Code Coverage"
| coverlet.collector instruments the assemblies under test
v
TestResults/<guid>/coverage.cobertura.xml (one per test project)
|
v
ReportGenerator: merge all files, produce HTML + summary
|
v
publish as a pipeline artefact or coverage tab, compare with main
| Collector | Command | Output |
|---|---|---|
Coverlet (coverlet.collector, in the xUnit template) |
--collect "XPlat Code Coverage" |
Cobertura XML |
| Microsoft Code Coverage | --collect "Code Coverage;Format=cobertura" |
Cobertura XML, or the binary .coverage format by default |
Configure includes and excludes (generated code, migrations, test projects) in a .runsettings file rather than on the command line. Treat coverage as a map of untested risk, not a number to hit.
| Option | What it does |
|---|---|
--blame |
Records which test was running when the host crashed |
--blame-crash |
Also collects a dump when the testhost crashes |
--blame-hang-timeout 10m |
If a single test runs longer than this, collects a dump and kills the host |
--blame-hang-dump-type full / mini |
Size and detail of the hang dump |
Without these, a hung test simply times out the whole CI job with no clue. With them, you get the test name and a dump you can open in Visual Studio or dotnet-dump.
| Level | Default | Control |
|---|---|---|
| Test projects | Run in parallel (MSBuild builds and tests projects concurrently) | -m:1 to serialise |
| Target frameworks of one project | Run in parallel | -p:TestTfmsInParallel=false |
| Tests within a project | Framework-specific: xUnit runs collections in parallel, NUnit and MSTest need opting in | xunit.runner.json, [Parallelizable], [assembly: Parallelize] |
Tests that share a database, a port or static state fail randomly when run in parallel. Isolate them (separate databases or containers per test class) rather than turning parallelism off everywhere.
A .runsettings file, passed with --settings tests.runsettings, keeps options in the repo:
| Section | Example settings |
|---|---|
RunConfiguration |
Results directory, target platform, timeouts, environment variables |
DataCollectionRunSettings |
Coverage collector and its include/exclude rules |
LoggerRunSettings |
Default loggers |
| Framework sections | Adapter-specific options |
A developer can then reproduce exactly what CI ran with the same file.
VSTest mode (default) MTP mode (opt in via global.json)
dotnet test --> vstest.console dotnet test --> runs each test project's
--> testhost per project own executable directly
--> adapter (the test framework IS the host)
| VSTest | Microsoft.Testing.Platform | |
|---|---|---|
| How tests run | Through vstest.console and a testhost | The test project is an executable |
| Select in .NET 10 | Default | "test": { "runner": "Microsoft.Testing.Platform" } in global.json |
| Results | --logger trx |
--report-trx (TrxReport extension) |
| Coverage | --collect ... |
--coverage (CodeCoverage extension) |
| Hang and crash dumps | --blame-hang-timeout, --blame-crash |
--hangdump --hangdump-timeout, --crashdump |
| Filters | VSTest --filter syntax |
Framework-specific; MSTest and NUnit accept --filter, xUnit v3 has its own --filter-* options |
| Frameworks | All | MSTest, NUnit, xUnit v3, TUnit |
Switch deliberately: confirm every test framework in the repo supports MTP, update pipeline arguments, and check your CI still picks up results.
--no-build after one solution build.Integration tests often need secrets. Get them from the pipeline's secret store as environment variables, never from a committed appsettings.Test.json. TRX files, console logs and blame dumps can contain data from test environments, so treat them as internal artefacts with limited retention.
Microsoft.Testing.Platform is where new test features are going. Watch the .NET 11 SDK notes for changes to the default runner before relying on them.
dotnet test.--filter "Category=Unit", then only integration tests.--logger trx --results-directory ./TestResults and open the TRX file.--collect "XPlat Code Coverage", find coverage.cobertura.xml, and turn it into HTML with ReportGenerator.Task.Delay(Timeout.Infinite) and run with --blame-hang-timeout 30s; find the dump and the name of the hung test.echo $? (bash) or $LASTEXITCODE (PowerShell)."test": { "runner": "Microsoft.Testing.Platform" } to global.json in a scratch copy with an MTP-enabled test project and compare the options.Create a test project and run it with filters, results and coverage:
dotnet new xunit -n Orders.Tests -o tests/Orders.Tests -f net10.0
dotnet test tests/Orders.Tests --filter "Category=Unit" --logger trx --results-directory ./TestResults
dotnet test tests/Orders.Tests --collect "XPlat Code Coverage" --blame-hang-timeout 5m
tests/Orders.Tests/PricingTests.cs:
namespace Orders.Tests;
public static class Pricing
{
public static decimal WithGst(decimal amount) => Math.Round(amount * 1.10m, 2, MidpointRounding.AwayFromZero);
}
public class PricingTests
{
[Fact]
[Trait("Category", "Unit")]
public void WithGst_adds_ten_percent() => Assert.Equal(110.00m, Pricing.WithGst(100m));
[Theory]
[Trait("Category", "Unit")]
[InlineData(0.05, 0.06)]
[InlineData(19.99, 21.99)]
public void WithGst_rounds_to_cents(decimal amount, decimal expected) =>
Assert.Equal(expected, Pricing.WithGst(amount));
[Fact]
[Trait("Category", "Integration")]
public async Task Integration_tests_are_filtered_separately()
{
await Task.Delay(10);
Assert.True(true);
}
}
Turn coverage into an HTML report (ReportGenerator as a local tool):
dotnet new tool-manifest
dotnet tool install dotnet-reportgenerator-globaltool
dotnet reportgenerator -reports:"tests/**/coverage.cobertura.xml" -targetdir:coverage-report -reporttypes:Html
Mistake: ignoring the exit code.
With continueOnError or a script that swallows errors, failing tests don't fail the build, so broken code ships.
Instead: let dotnet test fail the step, and publish results in a separate step that runs on success and failure.
Mistake: missing test packages.
A VSTest-based project needs Microsoft.NET.Test.Sdk plus the framework's adapter (xunit.runner.visualstudio, NUnit3TestAdapter or MSTest.TestAdapter). Without them dotnet test reports no tests, and the step can pass while running nothing.
Instead: create test projects from the dotnet new test templates, and check the test count in CI results.
Mistake: never separating slow tests from fast ones as the suite grows.
Every push waits for database and container tests, so feedback slows and developers stop running the suite.
Instead: tag tests with categories and run Category=Unit on every push and integration tests in a separate, parallel job.
Mistake: --no-build with the wrong configuration.
The test step looks in bin/Debug/ after a Release build and fails, or runs stale binaries.
Instead: pass the same -c to build and test.
Mistake: publishing test results only when tests pass.
The publish step is skipped after a failure, so the failures nobody can see are the ones that matter.
Instead: run the publish step with always() conditions.
Mistake: chasing a coverage percentage. Teams write tests that execute code without asserting anything, so the number rises while risk stays the same. Instead: use coverage to find untested risky code, review tests for meaningful assertions, and track coverage as a trend.
Mistake: no hang diagnostics.
A stuck test times out the whole job with no clue which test or why.
Instead: always run with --blame-hang-timeout (or --hangdump in MTP mode) so you get the test name and a dump.
Mistake: quietly re-running flaky tests until they pass. Flakiness hides real race conditions and destroys trust in the pipeline. Instead: track pass rates per test, quarantine flaky tests into a visible list with an owner, and fix them.
Mistake: tests that share state running in parallel. Tests using the same database, port or static field fail randomly depending on timing. Instead: isolate shared resources per test class or collection, rather than disabling parallelism for everything.
Mistake: secrets in committed test settings files. Anyone with repository access has credentials to test or shared environments. Instead: inject secrets as environment variables from the pipeline's secret store.
Mistake: mixing VSTest and Microsoft.Testing.Platform options after switching modes.
In MTP mode, --logger trx and --collect don't apply, so results and coverage silently disappear.
Instead: update the pipeline to MTP options such as --report-trx and --coverage when you switch, and verify the output.
Seniors treat the test command as part of the product's quality system, and they make the structure concrete:
dotnet test --filter "Category=Unit" across the solution, targeted at a few minutes. Test projects already run in parallel; within xUnit, keep collections independent so they can run in parallel too.--filter "Category=Integration", using containers (for example Testcontainers) for databases and brokers, run as a separate job so it doesn't block the fast feedback.--blame-hang-timeout on every run, and coverage published as a trend rather than a hard gate on a number.They keep the test infrastructure (.runsettings, filters, coverage settings) in the repo, so a developer can reproduce exactly what CI ran.
The pipeline restores three times, builds in Debug, runs dotnet format in a way that silently fixes files instead of failing, and publishes from a different build than the one it tested. I'd use global.json for the SDK, restore once with --locked-mode, build once in Release with --no-restore, then test and publish with --no-build. I'd run dotnet format --verify-no-changes so formatting drift fails the build, collect test results and coverage, and cache the NuGet folder.
Restore works out the full NuGet package graph, downloads packages to the global packages folder and writes obj/project.assets.json. Build compiles against those assets into bin/<Configuration>/<TFM>. Test runs the test projects against the built assemblies. Publish produces a deployable folder with the app, its dependencies and runtime config, in Release by default since .NET 8. Each later command restores and builds implicitly, so in CI I restore once and pass --no-restore and --no-build. That way every step uses exactly the same outputs and nothing is repeated. The configuration has to match across steps.
Last updated 1 Nov 2026.