Verify data in tests with ASP.NET Core and EF Core in memory

Introduction

You’re writing a Web API with ASP.NET Core 2.1, and use EF Core as your ORM.
If you follow the official guidance on doing integration tests in ASP.NET Core 2.1, then you can use either an in-memory database provider, or SQLite in-memory. We are using an in-memory database provider, which is setup in our CustomWebApplicationFactory as per the current Microsoft guidelines which are these:

public class CustomWebApplicationFactory<TStartup> 
    : WebApplicationFactory<RazorPagesProject.Startup>
{
    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        builder.ConfigureServices(services =>
        {
            // Create a new service provider.
            var serviceProvider = new ServiceCollection()
                .AddEntityFrameworkInMemoryDatabase()
                .BuildServiceProvider();

            // Add a database context (ApplicationDbContext) using an in-memory 
            // database for testing.
            services.AddDbContext<ApplicationDbContext>(options => 
            {
                options.UseInMemoryDatabase("InMemoryDbForTesting");
                options.UseInternalServiceProvider(serviceProvider);
            });

These work well when running integration tests, as it allows you to quickly setup a database for each test, independent of every other test.

The problem

I couldn’t find any guidance on how to verify the data gets written correctly to the database in an assertion. e.g. here is one of our tests. We are using the XBehave library:

[Scenario]
public void CreateApplication_ShouldReturn201()
{
    "Given a valid create application request"
      .x(() => _fixture.GivenAValidCreateApplicationRequest());
    "When an application is created"
      .x(() => _fixture.WhenAnApplicationIsCreated());
    "Then the response http status code is a 201"
      .x(() => _fixture.ThenTheResponseStatusCodeIs(HttpStatusCode.Created));
    "And the response should contain an id"
      .x(() => _fixture.ThenTheResponseShouldContainAnApplicationId());
    "And the database should contain the application"
      .x(() => _fixture.ThenTheDatabaseShouldContainTheApplication());
}

How to verify the final step – “ThenTheDatabaseShouldContainTheApplication”?

The solution

Firstly, you need to change the call to AddDbContext so that your DbContext is a Singleton (the default is Scoped).

services.AddDbContext<ApplicationDbContext>(options => 
{
    options.UseInMemoryDatabase("InMemoryDbForTesting");
    options.UseInternalServiceProvider(serviceProvider);
}, ServiceLifetime.Singleton);

Then, you can get a DbContext from the system under test by asking its service collection for one, with a helper property like so:

protected ApplicationDbContext DbContext
{
    get => TestServer.Host.Services.GetService(typeof(ApplicationDbContext)) as ApplicationDbContext;
}

Since the DbContext is a singleton it’ll be the same one the system under test used, so we can query the DbContext for it directly.

internal void ThenTheDatabaseShouldContainTheApplication()
{
    var application = DbContext.Applications.Find(ApplicationId);
    Assert.NotNull(application);
}
Advertisements

Azure Key Vault + MSI = failing Web API integration tests

Introduction

In a number of our ASP.NET Core Web APIs we’re using Azure Key Vault for keeping secrets such as connection strings and authentication credentials out of source control.

Initially we were using Azure Key Vault with a clientid and clientsecret, which are stored in appsettings.json like so:

  "KeyVault": {
    "Name": "MyApplicationDev",
    "ClientId": "abcdefg123-b292-4177-ba53-858227a9143c",
    "ClientSecret": "5m9g9cpuNc31abcJZcjkfP9/pDwJgQ+T82t/qCey7Nc="
  },

But this begs the question – what happens if the Key Vault ClientId and ClientSecret get compromised? To prevent this you can setup Key Vault to use Managed Service Identity (MSI). With that in place you don’t need to have the ClientId and ClientSecret in your appsettings.json, instead you only need the KeyVault url:

  "KeyVaultSettings": {
    "Url": "https://myapplicationdev.vault.azure.net"
  },

This is usually configured in the WebApi’s Program.cs, via something like:

public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
    WebHost.CreateDefaultBuilder(args)
        .ConfigureAppConfiguration((context, builder) =>
        {
            builder.SetBasePath(context.HostingEnvironment.ContentRootPath)
                    .AddEnvironmentVariables();

            var config = builder.Build();
            var tokenProvider = new AzureServiceTokenProvider();
            var keyvaultClient = new KeyVaultClient((authority, resource, scope)
               => tokenProvider.KeyVaultTokenCallback(authority, resource, scope));
                    
            builder.AddAzureKeyVault(config["KeyVaultSettings:Url"], keyvaultClient, new DefaultKeyVaultSecretManager());
        })
        .UseStartup<Startup>();

Under the covers, this will add an AzureKeyVaultConfigurationSource to the registered list of IConfigurationBuilders (as well as do other things).

The problem

In ASP.NET 2.1 we are using the new WebApplicationFactory<T> for running integration tests against an in-memory TestServer. When you run these tests locally it’ll use your (i.e. you, the developer’s) AD credentials to authenticate against the Key Vault, and so the tests should pass if you have access to the Key Vault. However, when the tests run on the Build server (as part of your CI pipeline) then they’ll probably fail because the Build agent does not have access to the Key Vault.

My solution

Firstly, I didn’t want to change the code of the system under test (i.e. the Web API), i.e. by adding configuration to determine whether to use Key Vault or not. Although thinking about it, that might have been easier! But it feels a bit dirty to change the application code to make integration tests easier.

The approach I took was to remove the AzureKeyVaultConfigurationSource from the list of IConfigurationBuilders from the system under test, in my custom WebApplicationFactory<T>, i.e:

protected override void ConfigureWebHost(IWebHostBuilder builder)
{
    builder.ConfigureAppConfiguration((_, configurationBuilder) =>
    {
        var keyVaultSource = configurationBuilder.Sources.FirstOrDefault(cs => cs.GetType().Name == "AzureKeyVaultConfigurationSource");
        if (keyVaultSource != null)
        {
            configurationBuilder.Sources.Remove(keyVaultSource);
        }
    });
}

This way, the code in the system under test is unchanged, instead we are just changing the configuration of the TestServer prior to it starting. Hope that helps someone.

Where to download vstest.console.exe

Often when I’m debugging builds or releases on VSTS, I will see that it uses the vstest.console.exe command line for running tests. Sometimes I need to run vstest.console.exe locally so that I can debug test run failures.

FYI, it is installed as part of Visual Studio 2017, and if you run the Visual Studio Developer Command prompt you will be able to run it from there. The executable lives in C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow

If you don’t have VS2017 installed and you need to download it so that you can run it locally, you can find it in the Microsoft.TestPlatform NuGet package.

Once you’ve downloaded the NuGet package, rename it from microsoft.testplatform.15.8.0.nupkg to microsoft.testplatform.15.8.0.zip. Then open it, and you’ll find vstest.console.exe in the tools\net451\Common7\IDE\Extensions\TestPlatform folder.

Add an authorization header to your swagger-ui with Swashbuckle (revisited)

Just over a year ago I blogged a simple way to add an authorization header to your swagger-ui with Swashbuckle. Although that works, Swagger-UI and Swashbuckle support a better way, which I’ll describe below.

Before starting I assume you’ve already got OAuth2 setup correctly on your application (using bearer tokens), and you have decorated your controllers and actions with [Authorize] attributes. If you haven’t, that is beyond the scope of this blog post. Here all I’m doing is explaining how to configure Swashbuckle.

First, you need to tell Swashbuckle what security your API has:

services.AddSwaggerGen(options =>
{
    options.AddSecurityDefinition("oauth2", new ApiKeyScheme
    {
        Description = "Standard Authorization header using the Bearer scheme. Example: \"bearer {token}\"",
        In = "header",
        Name = "Authorization",
        Type = "apiKey"
    });

This adds a securityDefinition to the bottom of the Swagger document, which Swagger-UI renders as an “Authorize” button:

Clicking that brings up a dialog box where you can put your bearer token:

The next thing we need to do is tell Swashbuckle which of our actions require Authorization. To do that you can use the SecurityRequirementsOperationFilter:

services.AddSwaggerGen(options =>
{
    options.AddSecurityDefinition("oauth2", new ApiKeyScheme
    {
        Description = "Standard Authorization header using the Bearer scheme. Example: \"bearer {token}\"",
        In = "header",
        Name = "Authorization",
        Type = "apiKey"
    });

    options.OperationFilter<SecurityRequirementsOperationFilter>();

You can either download the SecurityRequirementsOperationFilter from here, or, if you’re using ASP.NET Core you can install my Swashbuckle.AspNetCore.Filters package from NuGet, which includes it (and other filters).

The SecurityRequirementsOperationFilter adds a security property to each operation in the Swagger document, which renders in Swagger-UI as a padlock next to the operation:

Once you’ve done that, when you “Try it out” using the Swagger-UI, the authorization header with your bearer token should be sent to your API.

Show Swagger documentation on Azure Service Fabric

Another blog post in what seems to be becoming a series of posts on Swagger.

Swashbuckle allows you to include XML comments on your API’s Swagger page. To do this you need to set your Build to output an XML file, which Swashbuckle reads the XML comments from.
Don’t forget to add that XML file to the “Release” build configuration too, otherwise you won’t have XML comments when your application is actually deployed to an environment.

Here’s a handy code snippet for your .NET Core .csproj file:

  <!-- Begin Swagger documentation file-->
  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
    <DocumentationFile>bin\Debug\net46\win10-x64\swagger.xml</DocumentationFile>
    <NoWarn>1701;1702;1705;1591</NoWarn>
  </PropertyGroup>

  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
    <DocumentationFile>bin\Debug\net46\win10-x64\swagger.xml</DocumentationFile>
    <NoWarn>1701;1702;1705;1591</NoWarn>
  </PropertyGroup>

  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|AnyCPU'">
    <DocumentationFile>bin\Release\net46\win10-x64\swagger.xml</DocumentationFile>
    <NoWarn>1701;1702;1705;1591</NoWarn>
  </PropertyGroup>

  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
    <DocumentationFile>bin\Release\net46\win10-x64\swagger.xml</DocumentationFile>
    <NoWarn>1701;1702;1705;1591</NoWarn>
  </PropertyGroup>
  <!-- End Swagger documentation file-->

Additionally, if your application is deployed on Azure Service Fabric, you will also need to copy the XML file to your PublishDir, with the following snippet in your .csproj:

  <Target Name="PrepublishScript" BeforeTargets="PrepareForPublish">
    <ItemGroup>
      <DocFile Include="bin\x64\$(Configuration)\$(TargetFramework)\win10-x64\*.xml" />
    </ItemGroup>
    <Copy SourceFiles="@(DocFile)" DestinationFolder="$(PublishDir)" SkipUnchangedFiles="false" />
  </Target>

And of course you need to tell Swashbuckle to use that XML file via the .IncludeXmlComments() method mentioned above.

Fix slow NCrunch build

NCrunch is a great unit test runner and has been part of my developer toolbox for many years.

Recently I was having an issue on a medium-sized project where the NCrunch’s Build step was taking almost 2 minutes to complete. That spinning “B” was taking forever!

NCrunch progress

The tests themselves would run in under one minute.

The problem was we use a custom Autofixture [AutoData] attribute on every test.

Our attribute’s constructor creates a TestServer for running the web tests against.
We then inject the test server into each test, i.e.

[Scenario]
[WebHostAutoData]
public void CreateJob_ShouldCreateSuccessfully(
    WebApiClientFixture apiFixture

As it turns out, NCrunch enumerates all test attributes during Analysis, so you should not have any expensive code in your attribute’s constructor.

In my case, I fixed this by introducing a base class for all of my tests, which creates the TestServer (during test run). I then dropped our custom [AutoData] attribute.