📁 Sangam Dhakal - Portfolio v2.0 - Netscape Navigator 4.0
_
□
×

📚 Interview question

How does dotnet test work, and how do you filter tests, collect code coverage and publish results in CI?

Running tests from the CLI: how dotnet test discovers and runs tests, VSTest filters, loggers and TRX results, Coverlet and Microsoft code coverage, hang and crash diagnostics, parallelism, and Microsoft.Testing.Platform in .NET 10.

Intermediate Theory Intermediate ci-cd code-coverage dotnet-test unit-testing xunit

🎯 Key points

  • `dotnet test` builds test projects (unless --no-build) and runs them through the test platform with the project's test adapter
  • `--filter` selects tests by FullyQualifiedName, Name, Category/Trait and other properties, with & | and ~ operators
  • `--logger trx` (or junit via a logger package) writes results that CI can display
  • Coverage: `--collect "XPlat Code Coverage"` with coverlet.collector, or Microsoft's code coverage collector
  • `--blame-hang-timeout` and `--blame-crash` capture dumps when tests hang or crash
  • Test projects run in parallel by default; tests within a project follow the framework's parallelisation rules
  • In .NET 10 the SDK can run dotnet test in Microsoft.Testing.Platform mode, opted into via global.json
  • The exit code is non-zero on failure, so the pipeline must not ignore it

⏱ 30-second answer

dotnet test builds the test projects and runs them through the test platform, using the adapter the project references (xUnit, NUnit or MSTest). I use --filter to pick tests, for example --filter "Category=Unit&FullyQualifiedName~Orders". I use --logger trx so CI can show the results, and --collect "XPlat Code Coverage" with Coverlet for coverage, which I turn into a report. When tests hang or crash, --blame-hang-timeout and --blame-crash capture dumps. The command exits non-zero on failure, so the pipeline fails. In .NET 10 the SDK can also run in Microsoft.Testing.Platform mode, which you opt into in global.json.

🗣 Full answer (senior)

How I would answer this in an interview

"dotnet test builds the test projects, unless I pass --no-build after one solution build, and runs them through the test platform using the adapter each project references, xUnit, NUnit or MSTest, alongside Microsoft.NET.Test.Sdk. Test projects run in parallel by default, and within a project the framework's own rules decide. The exit code is non-zero on any failure, so the pipeline must never swallow it.

I use --filter to pick tests by FullyQualifiedName, name or category, with operators for equals, contains, and, and or, which lets me split fast unit tests from slower integration tests into separate stages. --logger trx writes results CI can display, or JUnit through a logger package, and when I wire a pipeline I publish them even when tests fail. For coverage I add --collect "XPlat Code Coverage" with coverlet.collector, or Microsoft's collector, and treat it as a signal for untested risk, not a target.

When a run hangs or crashes, --blame-hang-timeout and --blame-crash capture a dump, so a mysterious timed-out pipeline becomes something I can analyse. In .NET 10 the SDK can also run in Microsoft.Testing.Platform mode, opted into in global.json; its options differ, so the mistake I see is switching modes without updating the pipeline arguments and silently losing results and coverage. I keep run settings and filters in the repo, so anyone can reproduce exactly what CI ran."


What happens

  1. dotnet test on a solution or project builds the test projects (skip with --no-build).
  2. For each test project it starts a test host that loads the adapter (xunit.runner.visualstudio, NUnit3TestAdapter, MSTest.TestAdapter) referenced by the project, together with Microsoft.NET.Test.Sdk.
  3. The adapter discovers and runs tests; results flow back to the console and any loggers.
  4. The exit code is non-zero if any test fails or the run itself fails.

Test projects run in parallel by default; within a project, the framework decides (xUnit runs test collections in parallel, for example).

Filtering

--filter "FullyQualifiedName~Orders.Domain"          # contains
--filter "Category=Integration"                       # NUnit/MSTest category, xUnit [Trait("Category", ...)]
--filter "Category!=Slow&FullyQualifiedName~Pricing"  # and
--filter "Name=Total_includes_gst|Name=Total_rounds"  # or

Use filters to split fast unit tests from slower integration tests into separate pipeline stages once the full suite takes too long for every push.

Results and coverage

  • --logger trx --results-directory ./TestResults writes TRX files that Azure DevOps and most CI systems display; logger packages add JUnit output for others.
  • --collect "XPlat Code Coverage" uses coverlet.collector (included in the xUnit template) to write Cobertura XML; ReportGenerator turns it into HTML and summaries. Microsoft's own collector (--collect "Code Coverage") is another option.
  • Treat coverage as a signal for untested risk, not a target to game.

Hangs and crashes

--blame-hang-timeout 5m collects a dump and kills the test host if a test runs longer than the timeout; --blame-crash collects a dump if the host crashes. Both turn a mysterious timed-out pipeline into something you can analyse.

Microsoft.Testing.Platform in .NET 10

Microsoft.Testing.Platform (MTP) is the newer, lighter test platform: test projects built on it are executables you can run directly, and MSTest, NUnit, xUnit v3 and TUnit all support it. Historically dotnet test drove MTP projects through a VSTest bridge. The .NET 10 SDK adds a native MTP mode for dotnet test, which you opt into per repository in global.json:

{
  "sdk": { "version": "10.0.100" },
  "test": { "runner": "Microsoft.Testing.Platform" }
}

In that mode, options follow MTP's syntax (for example its own report and coverage extensions) rather than VSTest's --logger and --collect. VSTest remains the default and is fully supported, so switch a large solution deliberately: confirm every test framework in it supports MTP, update pipeline arguments, and check that your CI still picks up results.

Production example

A pipeline intermittently timed out after 60 minutes with no clue which test hung. Adding --blame-hang-timeout 10m produced a dump on the next occurrence, which showed an integration test waiting on a TaskCompletionSource that a mocked message bus never completed. The fix took ten minutes once the evidence existed.

🔍 What the interviewer is checking

The interviewer is checking practical CI testing skills, not just writing tests.

Green flags

  • Knows how dotnet test finds and runs tests via adapters.
  • Uses filters, TRX loggers, coverage collectors and blame options.
  • Talks about separating fast and slow suites and failing the pipeline correctly.

Red flags

  • Only runs tests from the IDE.
  • Treats coverage percentage as the goal.
  • Has never diagnosed a hanging test run.

📚 Learn in depth

1. The moving parts (VSTest mode, the default)

 dotnet test Orders.slnx -c Release
   |
   +-- build each test project (unless --no-build)
   |
   v
 test platform (vstest.console)
   |
   +-- for EACH test project and EACH target framework:
   |      start a separate testhost process
   |         loads Microsoft.NET.Test.Sdk + the adapter package
   |         (xunit.runner.visualstudio / NUnit3TestAdapter / MSTest.TestAdapter)
   |         adapter discovers tests by reflection, then runs them
   |
   +-- data collectors attach to the testhost
   |      coverage (coverlet / Microsoft Code Coverage), blame (dumps)
   |
   +-- loggers receive results
   |      console, trx, junit (via a logger package), html
   |
   v
 exit code: 0 = all passed, non-zero = a failure or a run error

The separate testhost process isolates tests from the runner. It is how collectors attach, how a crash in a test doesn't take down the runner, and how blame can dump a hung host.

2. Filter syntax in full

--filter "<property><operator><value>", combined with & (and), | (or) and parentheses.

Operator Meaning
= Exact match
!= Not equal
~ Contains
!~ Does not contain
Framework Supported properties
xUnit FullyQualifiedName, DisplayName, and any trait name, such as Category from [Trait("Category", "Unit")]
NUnit FullyQualifiedName, Name, Priority, TestCategory / Category
MSTest FullyQualifiedName, Name, ClassName, Priority, TestCategory

Examples:

--filter "Category=Unit"
--filter "FullyQualifiedName~Orders.Domain"
--filter "Category!=Slow&FullyQualifiedName~Pricing"
--filter "(Category=Unit|Category=Contract)&FullyQualifiedName!~Legacy"

Characters such as (, ), & and | inside a value must be escaped with \, and in a shell the whole filter must be quoted.

3. Results: loggers

Logger How Used by
Console Built in; --logger "console;verbosity=detailed" for more output Local runs, CI logs
TRX --logger trx --results-directory ./TestResults Azure DevOps PublishTestResults@2, most CI reporters
JUnit --logger junit with the JunitXml.TestLogger package GitHub Actions reporters, GitLab, Jenkins
HTML --logger html Quick human-readable output

Publish the result files as a pipeline step that runs even when tests fail (condition: always() in Azure Pipelines, if: always() in GitHub Actions), otherwise the failures you most need to see are never published.

4. Coverage

 dotnet test --collect "XPlat Code Coverage"
    |  coverlet.collector instruments the assemblies under test
    v
 TestResults/<guid>/coverage.cobertura.xml   (one per test project)
    |
    v
 ReportGenerator: merge all files, produce HTML + summary
    |
    v
 publish as a pipeline artefact or coverage tab, compare with main
Collector Command Output
Coverlet (coverlet.collector, in the xUnit template) --collect "XPlat Code Coverage" Cobertura XML
Microsoft Code Coverage --collect "Code Coverage;Format=cobertura" Cobertura XML, or the binary .coverage format by default

Configure includes and excludes (generated code, migrations, test projects) in a .runsettings file rather than on the command line. Treat coverage as a map of untested risk, not a number to hit.

5. Hangs and crashes

Option What it does
--blame Records which test was running when the host crashed
--blame-crash Also collects a dump when the testhost crashes
--blame-hang-timeout 10m If a single test runs longer than this, collects a dump and kills the host
--blame-hang-dump-type full / mini Size and detail of the hang dump

Without these, a hung test simply times out the whole CI job with no clue. With them, you get the test name and a dump you can open in Visual Studio or dotnet-dump.

6. Parallelism

Level Default Control
Test projects Run in parallel (MSBuild builds and tests projects concurrently) -m:1 to serialise
Target frameworks of one project Run in parallel -p:TestTfmsInParallel=false
Tests within a project Framework-specific: xUnit runs collections in parallel, NUnit and MSTest need opting in xunit.runner.json, [Parallelizable], [assembly: Parallelize]

Tests that share a database, a port or static state fail randomly when run in parallel. Isolate them (separate databases or containers per test class) rather than turning parallelism off everywhere.

7. Run settings

A .runsettings file, passed with --settings tests.runsettings, keeps options in the repo:

Section Example settings
RunConfiguration Results directory, target platform, timeouts, environment variables
DataCollectionRunSettings Coverage collector and its include/exclude rules
LoggerRunSettings Default loggers
Framework sections Adapter-specific options

A developer can then reproduce exactly what CI ran with the same file.

8. Microsoft.Testing.Platform mode (.NET 10)

 VSTest mode (default)                     MTP mode (opt in via global.json)
 dotnet test --> vstest.console            dotnet test --> runs each test project's
             --> testhost per project                      own executable directly
             --> adapter                   (the test framework IS the host)
VSTest Microsoft.Testing.Platform
How tests run Through vstest.console and a testhost The test project is an executable
Select in .NET 10 Default "test": { "runner": "Microsoft.Testing.Platform" } in global.json
Results --logger trx --report-trx (TrxReport extension)
Coverage --collect ... --coverage (CodeCoverage extension)
Hang and crash dumps --blame-hang-timeout, --blame-crash --hangdump --hangdump-timeout, --crashdump
Filters VSTest --filter syntax Framework-specific; MSTest and NUnit accept --filter, xUnit v3 has its own --filter-* options
Frameworks All MSTest, NUnit, xUnit v3, TUnit

Switch deliberately: confirm every test framework in the repo supports MTP, update pipeline arguments, and check your CI still picks up results.

9. Performance

  • Keep unit tests free of I/O so thousands run in seconds.
  • Put slow integration tests behind a filter and run them as a separate, parallel job.
  • Use --no-build after one solution build.
  • Watch the slowest tests in the TRX results and fix them before the suite becomes a bottleneck.

10. Security

Integration tests often need secrets. Get them from the pipeline's secret store as environment variables, never from a committed appsettings.Test.json. TRX files, console logs and blame dumps can contain data from test environments, so treat them as internal artefacts with limited retention.

11. .NET 11 awareness

Microsoft.Testing.Platform is where new test features are going. Watch the .NET 11 SDK notes for changes to the default runner before relying on them.

12. Try it yourself

  1. Create the project in the code example and run dotnet test.
  2. Run only unit tests with --filter "Category=Unit", then only integration tests.
  3. Add --logger trx --results-directory ./TestResults and open the TRX file.
  4. Add --collect "XPlat Code Coverage", find coverage.cobertura.xml, and turn it into HTML with ReportGenerator.
  5. Add a test that awaits Task.Delay(Timeout.Infinite) and run with --blame-hang-timeout 30s; find the dump and the name of the hung test.
  6. Make the test fail and check the exit code with echo $? (bash) or $LASTEXITCODE (PowerShell).
  7. Add "test": { "runner": "Microsoft.Testing.Platform" } to global.json in a scratch copy with an MTP-enabled test project and compare the options.

💻 Code

Create a test project and run it with filters, results and coverage:

dotnet new xunit -n Orders.Tests -o tests/Orders.Tests -f net10.0
dotnet test tests/Orders.Tests --filter "Category=Unit" --logger trx --results-directory ./TestResults
dotnet test tests/Orders.Tests --collect "XPlat Code Coverage" --blame-hang-timeout 5m

tests/Orders.Tests/PricingTests.cs:

namespace Orders.Tests;

public static class Pricing
{
    public static decimal WithGst(decimal amount) => Math.Round(amount * 1.10m, 2, MidpointRounding.AwayFromZero);
}

public class PricingTests
{
    [Fact]
    [Trait("Category", "Unit")]
    public void WithGst_adds_ten_percent() => Assert.Equal(110.00m, Pricing.WithGst(100m));

    [Theory]
    [Trait("Category", "Unit")]
    [InlineData(0.05, 0.06)]
    [InlineData(19.99, 21.99)]
    public void WithGst_rounds_to_cents(decimal amount, decimal expected) =>
        Assert.Equal(expected, Pricing.WithGst(amount));

    [Fact]
    [Trait("Category", "Integration")]
    public async Task Integration_tests_are_filtered_separately()
    {
        await Task.Delay(10);
        Assert.True(true);
    }
}

Turn coverage into an HTML report (ReportGenerator as a local tool):

dotnet new tool-manifest
dotnet tool install dotnet-reportgenerator-globaltool
dotnet reportgenerator -reports:"tests/**/coverage.cobertura.xml" -targetdir:coverage-report -reporttypes:Html

⚠ Common mistakes

  • Mistake: ignoring the exit code. With continueOnError or a script that swallows errors, failing tests don't fail the build, so broken code ships. Instead: let dotnet test fail the step, and publish results in a separate step that runs on success and failure.

  • Mistake: missing test packages. A VSTest-based project needs Microsoft.NET.Test.Sdk plus the framework's adapter (xunit.runner.visualstudio, NUnit3TestAdapter or MSTest.TestAdapter). Without them dotnet test reports no tests, and the step can pass while running nothing. Instead: create test projects from the dotnet new test templates, and check the test count in CI results.

  • Mistake: never separating slow tests from fast ones as the suite grows. Every push waits for database and container tests, so feedback slows and developers stop running the suite. Instead: tag tests with categories and run Category=Unit on every push and integration tests in a separate, parallel job.

  • Mistake: --no-build with the wrong configuration. The test step looks in bin/Debug/ after a Release build and fails, or runs stale binaries. Instead: pass the same -c to build and test.

  • Mistake: publishing test results only when tests pass. The publish step is skipped after a failure, so the failures nobody can see are the ones that matter. Instead: run the publish step with always() conditions.

  • Mistake: chasing a coverage percentage. Teams write tests that execute code without asserting anything, so the number rises while risk stays the same. Instead: use coverage to find untested risky code, review tests for meaningful assertions, and track coverage as a trend.

  • Mistake: no hang diagnostics. A stuck test times out the whole job with no clue which test or why. Instead: always run with --blame-hang-timeout (or --hangdump in MTP mode) so you get the test name and a dump.

  • Mistake: quietly re-running flaky tests until they pass. Flakiness hides real race conditions and destroys trust in the pipeline. Instead: track pass rates per test, quarantine flaky tests into a visible list with an owner, and fix them.

  • Mistake: tests that share state running in parallel. Tests using the same database, port or static field fail randomly depending on timing. Instead: isolate shared resources per test class or collection, rather than disabling parallelism for everything.

  • Mistake: secrets in committed test settings files. Anyone with repository access has credentials to test or shared environments. Instead: inject secrets as environment variables from the pipeline's secret store.

  • Mistake: mixing VSTest and Microsoft.Testing.Platform options after switching modes. In MTP mode, --logger trx and --collect don't apply, so results and coverage silently disappear. Instead: update the pipeline to MTP options such as --report-trx and --coverage when you switch, and verify the output.

🏛 Senior insight: trade-offs

Seniors treat the test command as part of the product's quality system, and they make the structure concrete:

  • Fast stage on every push: dotnet test --filter "Category=Unit" across the solution, targeted at a few minutes. Test projects already run in parallel; within xUnit, keep collections independent so they can run in parallel too.
  • Integration stage in parallel: --filter "Category=Integration", using containers (for example Testcontainers) for databases and brokers, run as a separate job so it doesn't block the fast feedback.
  • Evidence on failure: TRX results published to the pull request, --blame-hang-timeout on every run, and coverage published as a trend rather than a hard gate on a number.
  • Flakiness as a tracked metric: most CI systems (Azure DevOps test analytics, GitHub test reporters) show pass rate and duration per test over time. A test that fails intermittently gets quarantined into its own visible list with an owner and a deadline, not silently retried forever.

They keep the test infrastructure (.runsettings, filters, coverage settings) in the repo, so a developer can reproduce exactly what CI ran.

🔗 Follow-ups the interviewer may ask

  1. When: Candidate mentions trx loggers or coverage collection. Why: Checks how test results and coverage are wired into a real pipeline.
    How to answer

    The pipeline restores three times, builds in Debug, runs dotnet format in a way that silently fixes files instead of failing, and publishes from a different build than the one it tested. I'd use global.json for the SDK, restore once with --locked-mode, build once in Release with --no-restore, then test and publish with --no-build. I'd run dotnet format --verify-no-changes so formatting drift fails the build, collect test results and coverage, and cache the NuGet folder.

  2. When: Candidate runs dotnet test without saying what it builds. Why: Checks that the candidate knows dotnet test builds implicitly and how --no-build avoids testing a different build.
    How to answer

    Restore works out the full NuGet package graph, downloads packages to the global packages folder and writes obj/project.assets.json. Build compiles against those assets into bin/<Configuration>/<TFM>. Test runs the test projects against the built assemblies. Publish produces a deployable folder with the app, its dependencies and runtime config, in Release by default since .NET 8. Each later command restores and builds implicitly, so in CI I restore once and pass --no-restore and --no-build. That way every step uses exactly the same outputs and nothing is repeated. The configuration has to match across steps.

↩ Usually asked after

More in .NET CLI

Last updated 1 Nov 2026.

🌐 Sangam's Webring

⏪ Previous 🏠 Main Next ⏩
💬 Chat with me