Applying Entity Framework Migrations to a Docker Container

I’m going to run through how to deploy an API and a database into two separate Docker containers then apply Entity Framework migrations. This will create and populate the database with the correct schema and reference data. My idea was that EF migrations should be a straightforward way to initialise a database. It wasn’t that easy. I’m going to go through the failed attempts as I think they are instructive. I know that most people just want the answer – so if that’s you then just jump to the end and it’s there.

Environment

I’m using a .Net Core 3 API with Entity Framework Core and the database is MySQL. I’ll also touch on how you would do it with Entity Framework 6. The docker containers are Windows, though as it’s .Net Core and MySQL you could use Linux as well if needed.

The demo project is called Learning Analytics and it’s simple student management application. It’s just what I’m tinkering around with at the moment.

Deploying into Docker without migrations.

The DockerFile is

FROM mcr.microsoft.com/dotnet/core/aspnet:3.1-nanoserver-1903 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443

FROM mcr.microsoft.com/dotnet/core/sdk:3.1-nanoserver-1903 AS build
WORKDIR /src
COPY ["LearningAnalytics.API/LearningAnalytics.API.csproj", "LearningAnalytics.API/"]
RUN dotnet restore "LearningAnalytics.API/LearningAnalytics.API.csproj"
COPY . .
WORKDIR "/src/LearningAnalytics.API"
RUN dotnet build "LearningAnalytics.API.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "LearningAnalytics.API.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "LearningAnalytics.API.dll"]

and there is a docker-compose.yml file to bring up the API container above and the database ….

services:
  db:
    image: dockersamples/tidb:nanoserver-sac2016
    ports:
      - "49301:4000"

  app:
    image: learninganalyticsapi:dev
    build:
      context: .
      dockerfile: LearningAnalytics.API\Dockerfile
    ports:
      - "49501:80"
    environment:
      - "ConnectionStrings:LearningAnalyticsAPIContext=Server=db;Port=4000;Database=LearningAnalytics;User=root;SslMode=None;ConnectionReset=false;connect timeout=3600"     
    depends_on:
      - db

networks:
  default:
    external:
      name: nat

if I go to the directory containing docker-compose.yml file and run

docker-compose up -d

I’ll get the database and the api up. I can browse to the API at a test endpoint (the API is bound to port 49501 in the docker compose file)

http://localhost:49501/test

but if I try to access the API and get a list of students at

http://localhost:49501/api/student

then the application will crash because the database is blank. I haven’t done anything to populate it. I’m going to use migrations to do that.

Deploying into Docker with migrations – what doesn’t work

I thought it would be easy but it proved not to be.

Attempt 1 – via docker-compose

My initial thought was run the migrations as part of the docker-compose file using the command directive. So in the docker-compose file

  app:
    image: learninganalyticsapi:dev
    build:
      context: .
      dockerfile: LearningAnalytics.API\Dockerfile
    ports:
      - "49501:80"
    environment:
      - "ConnectionStrings:LearningAnalyticsAPIContext=Server=db;Port=4000;Database=LearningAnalytics;User=root;SslMode=None;ConnectionReset=false;connect timeout=3600"     
    depends_on:
      - db
	command: ["dotnet", "ef", "database update"]

The app server depends on the database (depends_on) so docker compose will bring them up in dependency order. However even though the app container comes up after the db container it, isn’t necessarily ‘ready’. The official documentation says

However, for startup Compose does not wait until a container is “ready” (whatever that means for your particular application) – only until it’s running.

So when I try to run entity framework migrations against the db container from the app container it fails. The db container isn’t ready and isn’t guaranteed to be either.

Attempt 2 – via interactive shell

I therefore thought I could do the same but run it afterwards via an interactive shell (details of an interactive shell is here). The idea being that I could wrap all this up in a PowerShell script looking like this

docker-compose up -d
docker exec learninganalytics_app_1 c:\migration\LearningAnalytics.Migration.exe

but this doesn’t work because

  1. the container doesn’t have the SDK installed as part of the base image so donet command isn’t available. This is resolvable
  2. EF core migrations needs the source code to run. We only have the built application in the container; as it should be. This sucks and isn’t resolvable

Attempt 3 – via the Startup class

I’m coming round to the idea that there is going to have to be some kind of code change in the application. I can apply migrations easily via C#. So in the startup class I could do

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
	using (var serviceScope = app.ApplicationServices.GetService<IServiceScopeFactory>().CreateScope())
	{
		var context = serviceScope.ServiceProvider.GetRequiredService<MyDatabaseContext>();
		context.Database.Migrate();
	}
	
	//.. other code
}	

Which does work but isn’t great. My application is going to apply migrations every time it starts – not very performant. I don’t like it.

Deploying into Docker with migrations – what does work

The resolution is a combination of the failed attempts. The principle is

  1. Provide a separate utility that can run migrations
  2. deploy this into the docker application container into it’s own folder
  3. run it after docker-compose
  4. wrap it up in a PowerShell script.

Ef Migration Utility

This is a simple console app that references the API. The app is

class Program
{
	static void Main(string[] args)
	{
		Console.WriteLine("Applying migrations");
		var webHost = new WebHostBuilder()
			.UseContentRoot(Directory.GetCurrentDirectory())
			.UseStartup<ConsoleStartup>()
			.Build();

		using (var context = (DatabaseContext) webHost.Services.GetService(typeof(DatabaseContext)))
		{
			context.Database.Migrate();
		}
		Console.WriteLine("Done");
	}
}

and the Startup class is a stripped down version of the API start up

public class ConsoleStartup
{
	public ConsoleStartup()
	{
		var builder = new ConfigurationBuilder()
			.AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
			.AddEnvironmentVariables();
		Configuration = builder.Build();
   }

	public IConfiguration Configuration { get; }

	public void ConfigureServices(IServiceCollection services)
	{
		services.AddDbContext<DatabaseContext>(options =>
		{
			options.UseMySql(Configuration.GetConnectionString("LearningAnalyticsAPIContext"));

		});
	}

	public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
	{
   
	}
}

I just need the Startup to read the app.config and get the database context up which this does. The console app references the API so it can use the API’s config files so I don’t have to double key the config into the console app.

DockerFile amends

The DockerFile file needs to be amended to deploy the migrations application into a separate folder on the app container file system. It becomes

FROM mcr.microsoft.com/dotnet/core/aspnet:3.1-nanoserver-1903 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443

FROM mcr.microsoft.com/dotnet/core/sdk:3.1-nanoserver-1903 AS build
WORKDIR /src
COPY ["LearningAnalytics.API/LearningAnalytics.API.csproj", "LearningAnalytics.API/"]
RUN dotnet restore "LearningAnalytics.API/LearningAnalytics.API.csproj"
COPY . .
WORKDIR "/src/LearningAnalytics.API"
RUN dotnet build "LearningAnalytics.API.csproj" -c Release -o /app/build

FROM mcr.microsoft.com/dotnet/core/sdk:3.1-nanoserver-1903 AS migration
WORKDIR /src
COPY . .
RUN dotnet restore "LearningAnalytics.Migration/LearningAnalytics.Migration.csproj"
COPY . .
WORKDIR "/src/LearningAnalytics.Migration"
RUN dotnet build "LearningAnalytics.Migration.csproj" -c Release -o /app/migration

FROM build AS publish
RUN dotnet publish "LearningAnalytics.API.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /migration
COPY --from=migration /app/migration .

WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "LearningAnalytics.API.dll"]

the relevant part is

FROM mcr.microsoft.com/dotnet/core/sdk:3.1-nanoserver-1903 AS migration
WORKDIR /src
COPY . .
RUN dotnet restore "LearningAnalytics.Migration/LearningAnalytics.Migration.csproj"
COPY . .
WORKDIR "/src/LearningAnalytics.Migration"
RUN dotnet build "LearningAnalytics.Migration.csproj" -c Release -o /app/migration

which builds out the migration application and …

FROM base AS final
WORKDIR /migration
COPY --from=migration /app/migration .

which copies it into a folder on the published container called migrations

Glue it together with PowerShell

Once the containers are brought up with docker-compose then it’s straightforward to use an interactive shell to navigate to the LearningAnalytics.Migration.exe application and run it. That will initialise the database. A better solution is to wrap it all up in a simple PowerShell script e.g.

docker-compose up -d
docker exec learninganalytics_app_1 c:\migration\LearningAnalytics.Migration.exe

and run that. The container comes up and the database is populated with the correct schema and reference data via EF migrations. The API now works correctly.

Entity Framework 6

The above is all for Entity Framework Core. Entity Framework 6 introduced the Migrate.exe tool . This can apply EF migrations without the source code which was the major stumbling block for EF Core. Armed with this then you could copy this up to the container and perform the migrations via something like

docker exec learninganalytics_app_1 Migration.exe

Do Migrations suck though?

This person thinks so. Certainly the inability to run them on compiled code is a huge drag. Whenever I write a production application then I prefer to just write the SQL out for the schema and apply it with some PowerShell. It’s not that hard. I like to use migrations for personal projects but there must be a reason that I’m not using them when I get paid to write code. Do I secretly think that they suck just a little?

Demo code

As ever, demo code is on my git hub site

https://github.com/timbrownls20/Learning-Analytics/tree/master/LearningAnalytics/LearningAnalytics.Migration
is the migration app

https://github.com/timbrownls20/Learning-Analytics/blob/master/LearningAnalytics/LearningAnalytics.API/DockerfileMigrations
the DockerFile

https://github.com/timbrownls20/Learning-Analytics/tree/master/LearningAnalytics
for the docker-compose.yml file and the simple PowerShell that glues it together

Useful links


This Stack Overflow question was the starting point for a lot of this and this answer particularly has a good discussion and some other options on how to achieve this – none of them are massively satisfactory. I felt something like what I’ve done was about the best.

https://docs.docker.com/compose/startup-order/
discusses why you can’t rely on the depends_on directive to make the database available to the application when you are bringing up the containers. It has more possibilities to circumvent this, such as wait-for-it. I’m certainly going to look at these but they do seem scoped to Linux rather than Windows so I’d have to change around the docker files for that. Also they wouldn’t help with Entity Framework 6 or earlier.

NuGet restore failing in Docker Container

I was tempted to write about this before, but I didn’t as there is already a very good, highly rated stack overflow answer with the solution. However, I’m just reinstalling Docker desktop and getting things working again and I wish I had written this stuff down as I’ve forgotten it. One of the many reasons to write blog posts is to fix stuff in my memory and as my own personal development notes. So in that spirit…

The Problem

We have a very simple .Net Core MVC solution.

It has the following NuGet packages

Install-Package NugetSample.NugetDemo.Demo -Version 1.0.0
Install-Package bootstrap -Version 4.5.0

With this DockerFile to containerise it

FROM mcr.microsoft.com/dotnet/core/aspnet AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443

FROM mcr.microsoft.com/dotnet/core/sdk
WORKDIR /src
COPY ["Template.Web.csproj", "Template.Web/"]
RUN dotnet restore "Template.Web/Template.Web.csproj"
COPY . .
WORKDIR "/src/Template.Web"
RUN dotnet build "Template.Web.csproj" -c Release -o /app

FROM build AS publish
RUN dotnet publish "Template.Web.csproj" -c Release -o /app

FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "Template.Web.dll"]

We go to the directory with the DockerFile and try to build it into a container with

docker build .

It fails on the dotnet restore step like so …

i.e. with this error

C:\Program Files\dotnet\sdk\3.1.302\NuGet.targets(128,5): error : Unable to load the service index for source https://api.nuget.org/v3/index.json. [C:\src\Template.Web\Template.Web.csproj]
C:\Program Files\dotnet\sdk\3.1.302\NuGet.targets(128,5): error :   No such host is known. [C:\src\Template.Web\Template.Web.csproj]
The command 'cmd /S /C dotnet restore "Template.Web/Template.Web.csproj"' returned a non-zero code: 1

NuGet is failing us

The Cause

The container doesn’t have connectivity to the internet so can’t get bring down the packages. We can see this clearly by building this very very simple docker file

FROM mcr.microsoft.com/dotnet/core/sdk
RUN ping google.com

The ping fails. The host (my development machine) does have internet access – I would have noticed if that had gone down and I would be hysterically ringing Telstra (again). So it’s something specific to the container.

The Resolution

The DNS server is wrong in the container. To fix, hardcode the DNS into Docker i.e. put this JSON

"dns": ["10.1.2.3", "8.8.8.8"]

into the Docker daemon settings. In Docker Desktop it’s here

And restart the docker service. The container now has internet access, NuGet restore will work and we can now containerise our very simple web application.

Demo Code

As ever, the demo code is on my GitHub site

The very simple application
https://github.com/timbrownls20/Demo/tree/master/ASP.NET%20Core/Template

and its docker file
https://github.com/timbrownls20/Demo/blob/master/ASP.NET%20Core/Template/Template.Web/Dockerfile

Docker file for the internet test
https://github.com/timbrownls20/Demo/blob/master/Docker/InternetTest/DockerFile

Useful Links

This Stack Overflow answer has the resolution to this with a very good explanation. Also it has other (probably better) ways to fix this and resolutions to other Docker network issues that you may face.

Blocked by CORS policy? Unblocking in .Net Core 3

A while ago I wrote an post about hosting Angular under an existing IIS website. Quite a few people seem to have found it useful which is good. My motivation was to avoid CORS policy errors i.e blocked JavaScript requests to a different domain. I bypassed them completely by hosting the Angular client and the UI under the same domain – bit of a cheat really. At the time I wrote

This is to avoid Cross Origin Scripting Issues in my own environment. [.. ]other ways to do this

Never worked
Were too invasive to implement on the API side
Did work then frustratingly stopped working

I don’t know why I was struggling so much. It turns out to be pretty straight forward to have a CORS policy that lets anything through. I suspect its got a lot easier in .Net Core 3. Perhaps I just missed it before.

Cross-origin resource sharing (CORS)

Just to define terms- CORS is a way to enable one website to access resources on another domain. Often requests are blocked if they are from a different host (same-origin policy). It’s typically when JavaScript clients (Angular, React etc..) make a request to a API on a different host using XMLHttpRequest. When this happens, we see something like

blocked by CORS policy

In this case we need a suitable CORS Policy.

Enabling CORS for all hosts in .Net Core

Here I’m going to create a very relaxed CORS policy; it’s going to let anything through. In Startup.cs file ConfigureServices add

public void ConfigureServices(IServiceCollection services)
{
    //.. configure other services
    
    services.AddCors(options =>
    {
        options.AddPolicy("AllOrigins",
            builder =>
            {
                builder.AllowAnyHeader()
                               .AllowAnyOrigin()
                              .AllowAnyMethod();
            });
    });
}

and wire it up in the Configure method

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{	
    //.. more code

    app.UseCors("AllOrigins");

}

and that’s it. The error goes away and any JavaScript client can make a request against this API. This is good for my development projects but if this was to go into production you’d want to consider a finer tuned CORS policy. Same-origin policy is implemented in browsers for a reason, not just to frustrate my demo projects.

Demo code

As ever, the full code is at on my github site here.

Useful Links

https://en.wikipedia.org/wiki/Cross-origin_resource_sharing#Headers
Wikipedia has a good page on Cross-origin resource sharing

https://code.google.com/archive/p/browsersec/wikis/Part2.wiki#Same-origin_policy.
I found the same-origin policy browser security notes from Google interesting. Same-origin policy is a bit of an umbrella terms for some related security concerns. The ones that is causing the problem here is the same-origin policy for XMLHttpRequest but there are same-origin policy for DOM access, cookies and Java (as well as Silverlight and Flash – remember those guys anyone?)

Dynamically Loading Assemblies for Dependency Injection in .Net Core

We want to get all our assemblies registered for Dependency Injection in a .Net Core 3 application by scanning the bin folder and registering all the classes which implement a specific interface. Bit of a mouthful so a simple (and classic) example is

public class Foo: IFoo
{
}

public interface IFoo : ILifecycle
{
}
public class Bar: IBar
{
}

public interface IBar: ILifecycle
{
}
public class Bar: IBar
{
}

public interface IBar: ILifecycle
{
}

We could register each object by its own interface i.e.

services.AddTransient<IFoo, Foo>();
services.AddTransient<IBar, Bar>();

That’s fine for 2 but for hundreds it will be a pain. We want to register the assemblies based on an interface that they all inherit – in this case ILifecycle. Also, we want to register all the assemblies in a given folder, typically the bin folder. This is what didn’t work in .Net Core for me and hence the post – no answer on the internet (that I could find) so I’m filling the internet gap.

Implementation

We will use Scrutor as our DI framework. It’s an extension of the native .Net DI framework with extensions for scanning and registering assemblies en mass.

Incorrect Implementation – LoadFromFile

Using an MVC application and within Startup

public void ConfigureServices(IServiceCollection services)
{
//..lots of other code

foreach (string assemblyPath in Directory.GetFiles(System.AppDomain.CurrentDomain.BaseDirectory, "*.dll", SearchOption.AllDirectories))
{
  var assembly = Assembly.LoadFile(assemblyPath);
  assemblies.Add(assembly);
}

//.. register
services.Scan(scan => scan
   .FromAssemblies(assemblies)
   .AddClasses(classes => classes.AssignableTo<ILifecycle>(), publicOnly: false)
   .AsImplementedInterfaces()
   .WithTransientLifetime());

//.. test
var provider = services.BuildServiceProvider();
var foo = provider.GetService<IFoo>(); //.. returns null
var bar = provider.GetService<IBar>(); //.. return null
}

Frustratingly this doesn’t work and it’s not obvious why. The test returns null Foo and Bar objects even though we have registered them in the correct way i.e. by scanning for ILifecycle and registering the object as their matching interface. I we debug and look at the loaded assemblies everything looks fine. The assemblies have loaded and I can use reflection to poke into them further and satisfy myself that they are indeed the correct assemblies. Except they aren’t

Correct Implementation – AssemblyLoadContext

public void ConfigureServices(IServiceCollection services)
{
//..lots of other code

foreach (string assemblyPath in Directory.GetFiles(System.AppDomain.CurrentDomain.BaseDirectory, "*.dll", SearchOption.AllDirectories))
{
var assembly = System.Runtime.Loader.AssemblyLoadContext.Default.LoadFromAssemblyPath(assemblyPath);
assemblies.Add(assembly);
}

//.. register
services.Scan(scan => scan
.FromAssemblies(assemblies)
.AddClasses(classes => classes.AssignableTo<ILifecycle>(), publicOnly: false)
.AsImplementedInterfaces()
.WithTransientLifetime());

//.. test
var provider = services.BuildServiceProvider();
var foo = provider.GetService<IFoo>(); //.. returns null
var bar = provider.GetService<IBar>(); //.. return null
}

Very similar except we have swapped

Assembly.LoadFile(assemblyPath);

for

System.Runtime.Loader.AssemblyLoadContext.Default.LoadFromAssemblyPath(assemblyPath);

Now it works and the DI framework can provide us with the correct objects based on their matching interface. All is good.

Explanation

I don’t really have one so if anyone can post a comment explain this then that would be very interesting. I do know that .Net Core handles its dependencies differently to .net framework i.e. it’s all NuGet packages so it makes sense that we need another way to correctly load the assemblies with their dependencies. What’s puzzling is that Assembly.LoadFromFile is still there and appears to work – except it doesn’t for DI.

Other DI Frameworks

I wasn’t exhaustive but I tried the with StructureMap and AutoFac and got the same issue. So, it’s not limited to Scrutor. You’d face these problems with other frameworks as well.

Useful Links

Stack Overflow question on How to load assemblies located in a folder in .net core console app. It’s where I found the proper way to load assemblies

Git Hub for Scutor. Works well (when you get over the assembly loading issue).

Converting .Net Standard Libraries to .Net Core

I was looking for a quick answer to this on the internet and couldn’t find it. So here it is

Open your .Net Standard csproj file in notepad or similar text editor. You’ll see this structure

<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netstandard2.0</TargetFramework>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\..\..\Framework\AnotherLibrary.csproj" />
</ItemGroup>
</Project>

Change the target framework to the desired version of core

<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp3.1</TargetFramework>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\..\..\Framework\AnotherLibrary.csproj" />
</ItemGroup>
</Project>

Save it
Reload the project in visual studio or other IDE
Done

(I know there will be all kinds of other complexities to this but I’ve just got a bunch of standard libraries that need to reference a core project and can’t. So, they all need a rapid upgrade to .Net Core – like so).

.Net Core 2.0: project.assets.json’ doesn’t have a target for ‘.NETStandard,Version=v2.0’

The Problem

Odd error when building a .net core project with Visual Studio Code. This built a few weeks ago – went back it it and I’m getting this

C:\Program Files\dotnet\sdk\2.0.0\Sdks\Microsoft.NET.Sdk\build\Microsoft.PackageDependencyResolution.targets(165,5): error : Assets file ‘myproject\obj\project.assets.json’ doesn’t have a target for ‘.NETStandard,Version=v2.0’. Ensure that restore has run and that you have included ‘netstandard2.0’ in the TargetFrameworks for your project. [myproject\MyProject.csproj]

.Net Core has been installed, it used to work and there is not much on the Internet. Bafflement.

Solution

Turns out the bin and obj folders for each of the project need to be manually deleted. Once done then the command

dotnet build

gave me the green light and normal service has been resumed. No idea what it going on but sorted now – so I thought I would take a minute and post the solution to my small corner of the Internet. Cheers.

Multiple Projects with .Net Core and Visual Studio Code

The challenge is to create a solution in VS Code with multiple projects. There are three projects

  1. A console application
  2. A web api application
  3. A library with data access code that is common to both.

It will look like this when we are finished

Expanded to show detail

We need a solution that can build out all these projects in one click. Since we using VS Code and enjoying it’s lightweightness we’ll be relying completely on the new .Net Core Command Line Interface. I enjoy a challenge so using the command line seems fun (to me). Let’s go.

(TL;DR;)

In case you don’t want to read the detail – the commands used to set up all projects, solutions and references are

From project root

dotnet new console -o Demo.ConsoleApp

dotnet new classlib -o Demo.Common

cd Demo.ConsoleApp

dotnet add reference ../Demo.Common/Demo.Common.csproj

dotnet add package MyPackage

cd ../

dotnet new webapi -o Demo.Api

cd Demo.Api

dotnet add reference ../Demo.Common/Demo.Common.csproj

cd ../

dotnet new sln -n Demo

dotnet sln add Demo.ConsoleApp/Demo.ConsoleApp.csproj

dotnet sln add Demo.Api/Demo.Api.csproj

dotnet build

Preliminary

To do this you’ll need VS Code’s C# extension installed into visual studio. Go to extensions and search for C#.

Install and away you go.

Note on command line

All .Net Core CLI commands can be entered into the terminal window in VS Code or into the standard command prompt if you prefer. Navigate to the root folder and enter your first command

Create Console Application

dotnet new console -o Demo.ConsoleApp

This installs our console application project. Since we are on .Net Core 2.0 we get the csproj file rather than the json project file of earlier versions. This installs the console app in a new folder with the same name as it’s containing folder. If we want a different name to the folder we do that by

dotnet new console -n MyProjectName -o Demo.ConsoleApp

But we don’t want that. We are keeping it simple. What I like about .Net Core is that it’s had all the gubbins stripped out. So our proj file for the console app is simply …

<Project Sdk="Microsoft.NET.Sdk">
  <ItemGroup>
    <ProjectReference Include="..\Demo.Common\Demo.Common.csproj" />
  </ItemGroup>
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp2.0</TargetFramework>
  </PropertyGroup>
</Project>

Nice. I understand that.

Create Common Library

Next we want to create a C# dll to host some common code like data access.

dotnet new classlib -o Demo.Common

Creates an empty project called Demo.Common with a nice simple proj file thus

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
  </PropertyGroup>
</Project>

Link Common Library to Console app

Change to the console app direct and run the add reference common

cd Demo.ConsoleApp
dotnet add reference ../Demo.Common/Demo.Common.csproj

Inserts the following into the console proj file

  <ItemGroup>
    <ProjectReference Include="..\Demo.Common\Demo.Common.csproj" />
 </ItemGroup>

So the console app can access code in the common project and the two will build out together.

Add Nuget package to Common Library

We commonly want to add some nuget packages to our projects. So lets add the HtmlAgility pack and a C# driver for MongoDB – no reason for them other than I often use them.

Change directories back to common library

 cd ../Demo.Common

Now add the packages

dotnet add package MongoDB.Driver
dotnet add package HtmlAgilityPack

Look at the proj file and you can see the package references inserted

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="HtmlAgilityPack" Version="1.6.5" />
    <PackageReference Include="MongoDB.Driver" Version="2.4.4" />
  </ItemGroup>
</Project>

VS Code likes to tell me to restore the project I.e. run dotnet restore. You don’t need to. Dotnet build or dotnet run does it for you.

Build Console Application

Let’s check things are working as they should and build our console app.

cd ../Demo.ConsoleApp
dotnet build

Output

  Demo.Common -> C:\development\Demo\ASP.NET Core\MultipleProjects\Demo.Common\bin\Debug\netstandard2.0\Demo.Common.dll

Demo.ConsoleApp -> C:\development\Demo\ASP.NET Core\MultipleProjects\Demo.ConsoleApp\bin\Debug\netcoreapp2.0\Demo.ConsoleApp.dll

Build succeeded.

0 Warning(s)

0 Error(s)

Good – by building the project we get both common and the console application building.

Create API 

Return to root folder

cd ../

Now add a web api project called Demo.Api

dotnet new webapi -o Demo.Api

Change to the Api folder

cd Demo.Api

Now add a reference to our common library

dotnet add reference ../Demo.Common/Demo.Common.csproj

The Api proj file looks a bit more complex but I’ll still pretty minimal compared to Visual Studio proj files.

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>netcoreapp2.0</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <Folder Include="wwwroot\" />
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.All" Version="2.0.0" />
  </ItemGroup>
  <ItemGroup>
    <DotNetCliToolReference Include="Microsoft.VisualStudio.Web.CodeGeneration.Tools" Version="2.0.0" />
  </ItemGroup>
  <ItemGroup>
    <ProjectReference Include="..\Demo.Common\Demo.Common.csproj" />
  </ItemGroup>
</Project>

Now let’s build that project. Staying in the Api folder

dotnet build

And the output shows us two projects have been built.

  Demo.Common -> C:\development\Demo\ASP.NET Core\MultipleProjects\Demo.Common\bin\Debug\netstandard2.0\Demo.Common.dll

Demo.Api -> C:\development\Demo\ASP.NET Core\MultipleProjects\Demo.Api\bin\Debug\netcoreapp2.0\Demo.Api.dll

Build succeeded.

0 Warning(s)

0 Error(s)

Joining everything together

Ok , so we want all the projects to build in one click. Because we have two independent applications (an API and a console app) we aren’t going to be able to do the build by navigating to a folder and building the proj file. We need a solution file to bind them all together

(this shamefully baffled me for a while. I was trying all sorts with launch.json etc… The solution is obvious though – embarrassed for not getting it immediately)

Back to root

cd ../

Now add the solution file. We don’t want it in it’s own directory so omit the -o parameter. Add in the -n otherwise it takes it’s name from the containing folder which we don’t want in this case.

dotnet new sln -n Demo

Now include the API and Console app

dotnet sln add Demo.ConsoleApp/Demo.ConsoleApp.csproj
dotnet sln add Demo.Api/Demo.Api.csproj

The solution file now includes the project info

Microsoft Visual Studio Solution File, Format Version 12.00

# Visual Studio 15

VisualStudioVersion = 15.0.26124.0

MinimumVisualStudioVersion = 15.0.26124.0

Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Demo.ConsoleApp", "Demo.ConsoleApp\Demo.ConsoleApp.csproj", "{FDFFDD15-D0C0-4D66-9694-B21D85DBA5AE}"

EndProject

Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Demo.Api", "Demo.Api\Demo.Api.csproj", "{C1B9391B-07FC-403A-AA46-41128BC1FD07}"

EndProject

Staying in the solution directory run build

dotnet build

And now all three projects are built.

Demo.Common -> C:\development\Demo\ASP.NET Core\MultipleProjects\Demo.Common\bin\Debug\netstandard2.0\Demo.Common.dll

Demo.ConsoleApp -> C:\development\Demo\ASP.NET Core\MultipleProjects\Demo.ConsoleApp\bin\Debug\netcoreapp2.0\Demo.ConsoleApp.dll

Demo.Api -> C:\development\Demo\ASP.NET Core\MultipleProjects\Demo.Api\bin\Debug\netcoreapp2.0\Demo.Api.dll

Build succeeded.

0 Warning(s)

0 Error(s)

Finished!!!

Demo code

The rather uninspiring skeleton demo app is at
https://github.com/timbrownls20/Demo/tree/master/ASP.NET%20Core/MultipleProjects

More interesting the same pattern is on a .Net Core API for some Buddhist texts I’ve been working on
https://github.com/timbrownls20/Pali-Canon/tree/master/Api

So

https://github.com/timbrownls20/Pali-Canon/tree/master/Api/PaliCanon.Loader
Is the console app that parses HTML and populates a document database with the texts

https://github.com/timbrownls20/Pali-Canon/tree/master/Api/PaliCanon.Api
Is the web api to access the texts

https://github.com/timbrownls20/Pali-Canon/tree/master/Api/PaliCanon.Common
Is the model and repositories common to both.  It’s the same pattern as the simplified demo app and I’m using VS Code for that – just to try it out really.

Useful Links

https://docs.microsoft.com/en-us/dotnet/core/tools/?tabs=netcore2x
Full documentation for .Net Core Command Line Interface

https://code.visualstudio.com/download
Visual Studio Code is freely available here.

Getting the Root Directory Path for .Net Core Applications

Quick one. Given any .Net app I want to know the root path of the application i.e. the top level directory. I was doing this with .Net Core but it puts in extra directories by default in the exe path i.e.

bin\Debug\netcoreapp2.0\my.dll

so my normal kuldgy methods weren’t working. This all works for any .Net app not just Core in fairness – it’s just that the extra stuffed in by .Net Core threw me a little. There are all kinds of ways to find this for particular applications web, windows etc.. but this should work generally. Well it worked with my little console app anyway.

The Code

Use reflection to targeting the executable and get the path

var rootDir  = System.Reflection.Assembly.GetExecutingAssembly().CodeBase;

this will get you

file:///c:/development/MyApp/bin/Debug/netcoreapp2.0/myApp.dll

which is the UNC path. Combine with Path.GetDirectoryName i.e.

var rootDir  = System.IO.Path.GetDirectoryName(System.Reflection.Assembly.
GetExecutingAssembly().CodeBase);

this gets

file:\c:\development\MyApp\bin\Debug\netcoreapp2.0

which is a  file path now and it has got us the directory of the executable. It’s still not where we want to be so use some regex

(?<!fil)[A-Za-z]:\\+[\S\s]*?(?=\\+bin)

This will chop up the path and gives you

c:\development\MyApp

which is correct.

The Function

So overall a useful function for this could be

public string GetApplicationRoot()
{
 var exePath =   Path.GetDirectoryName(System.Reflection
                   .Assembly.GetExecutingAssembly().CodeBase);
 Regex appPathMatcher=new Regex(@"(?<!fil)[A-Za-z]:\\+[\S\s]*?(?=\\+bin)");
 var appRoot = appPathMatcher.Match(exePath).Value;
 return appRoot;
}

Which does what we need or I quite like it as a (much abused) extension method

public static string ToApplicationPath(this string fileName)
{
 var exePath = Path.GetDirectoryName(System.Reflection
                     .Assembly.GetExecutingAssembly().CodeBase);
 Regex appPathMatcher=new Regex(@"(?<!fil)[A-Za-z]:\\+[\S\s]*?(?=\\+bin)");
 var appRoot = appPathMatcher.Match(exePath).Value;
 return Path.Combine(appRoot, fileName);
}

Which can be called like

“TargetFile.cs”.ToApplicationPath()

Which gives

c:\development\MyApp\TargetFile.cs

Job done.

The Regex

Not everyone likes Regex*. But for the aficionados here is what it is doing

(?<!fil)[A-Za-z]:\\+[\S\s]*?(?=\\+bin)

For example matching this

file:\c:\development\MyApp\bin\Debug\netcoreapp2.0

[A-Za-z]:\\+[\S\s]*?

Is my match – this will match a drive and a path up to

(?=\\+bin)

It’s a lookahead (zero length assertion). It will check for “bin” but not include it in the match

(?<!fil)

It’s a negative lookbehind (zero length assertion again). It will make sure that the letters fil don’t immediately proceed the match. This is to stop e:// being the match i.e. the e from file.

Clearly this assumes that the bin folder is directly under the root path as it normal is. If it isn’t in your app then break out your favourite Regex editor and write your own. It will make a man/woman/scared child of you.

(* Hardly anyone likes it – well I haven’t met many)

Useful Links

https://stackoverflow.com/questions/837488/how-can-i-get-the-applications-path-in-a-net-console-application
Full disclosure – the first bit of this is from this SO question. The extension I did with my very own coding fingers

http://regexstorm.net/tester
Regex storm – C# regex tester. Very good. JavaScript based testers don’t work with this as they can’t do negative lookbehinds.

https://www.regular-expressions.info/lookaround.html
Look ahead, behind and all around with zero length assertions