Applying Entity Framework Migrations to a Docker Container

I’m going to run through how to deploy an API and a database into two separate Docker containers then apply Entity Framework migrations. This will create and populate the database with the correct schema and reference data. My idea was that EF migrations should be a straightforward way to initialise a database. It wasn’t that easy. I’m going to go through the failed attempts as I think they are instructive. I know that most people just want the answer – so if that’s you then just jump to the end and it’s there.

Environment

I’m using a .Net Core 3 API with Entity Framework Core and the database is MySQL. I’ll also touch on how you would do it with Entity Framework 6. The docker containers are Windows, though as it’s .Net Core and MySQL you could use Linux as well if needed.

The demo project is called Learning Analytics and it’s simple student management application. It’s just what I’m tinkering around with at the moment.

Deploying into Docker without migrations.

The DockerFile is

FROM mcr.microsoft.com/dotnet/core/aspnet:3.1-nanoserver-1903 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443

FROM mcr.microsoft.com/dotnet/core/sdk:3.1-nanoserver-1903 AS build
WORKDIR /src
COPY ["LearningAnalytics.API/LearningAnalytics.API.csproj", "LearningAnalytics.API/"]
RUN dotnet restore "LearningAnalytics.API/LearningAnalytics.API.csproj"
COPY . .
WORKDIR "/src/LearningAnalytics.API"
RUN dotnet build "LearningAnalytics.API.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "LearningAnalytics.API.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "LearningAnalytics.API.dll"]

and there is a docker-compose.yml file to bring up the API container above and the database ….

services:
  db:
    image: dockersamples/tidb:nanoserver-sac2016
    ports:
      - "49301:4000"

  app:
    image: learninganalyticsapi:dev
    build:
      context: .
      dockerfile: LearningAnalytics.API\Dockerfile
    ports:
      - "49501:80"
    environment:
      - "ConnectionStrings:LearningAnalyticsAPIContext=Server=db;Port=4000;Database=LearningAnalytics;User=root;SslMode=None;ConnectionReset=false;connect timeout=3600"     
    depends_on:
      - db

networks:
  default:
    external:
      name: nat

if I go to the directory containing docker-compose.yml file and run

docker-compose up -d

I’ll get the database and the api up. I can browse to the API at a test endpoint (the API is bound to port 49501 in the docker compose file)

http://localhost:49501/test

but if I try to access the API and get a list of students at

http://localhost:49501/api/student

then the application will crash because the database is blank. I haven’t done anything to populate it. I’m going to use migrations to do that.

Deploying into Docker with migrations – what doesn’t work

I thought it would be easy but it proved not to be.

Attempt 1 – via docker-compose

My initial thought was run the migrations as part of the docker-compose file using the command directive. So in the docker-compose file

  app:
    image: learninganalyticsapi:dev
    build:
      context: .
      dockerfile: LearningAnalytics.API\Dockerfile
    ports:
      - "49501:80"
    environment:
      - "ConnectionStrings:LearningAnalyticsAPIContext=Server=db;Port=4000;Database=LearningAnalytics;User=root;SslMode=None;ConnectionReset=false;connect timeout=3600"     
    depends_on:
      - db
	command: ["dotnet", "ef", "database update"]

The app server depends on the database (depends_on) so docker compose will bring them up in dependency order. However even though the app container comes up after the db container it, isn’t necessarily ‘ready’. The official documentation says

However, for startup Compose does not wait until a container is “ready” (whatever that means for your particular application) – only until it’s running.

So when I try to run entity framework migrations against the db container from the app container it fails. The db container isn’t ready and isn’t guaranteed to be either.

Attempt 2 – via interactive shell

I therefore thought I could do the same but run it afterwards via an interactive shell (details of an interactive shell is here). The idea being that I could wrap all this up in a PowerShell script looking like this

docker-compose up -d
docker exec learninganalytics_app_1 c:\migration\LearningAnalytics.Migration.exe

but this doesn’t work because

  1. the container doesn’t have the SDK installed as part of the base image so donet command isn’t available. This is resolvable
  2. EF core migrations needs the source code to run. We only have the built application in the container; as it should be. This sucks and isn’t resolvable

Attempt 3 – via the Startup class

I’m coming round to the idea that there is going to have to be some kind of code change in the application. I can apply migrations easily via C#. So in the startup class I could do

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
	using (var serviceScope = app.ApplicationServices.GetService<IServiceScopeFactory>().CreateScope())
	{
		var context = serviceScope.ServiceProvider.GetRequiredService<MyDatabaseContext>();
		context.Database.Migrate();
	}
	
	//.. other code
}	

Which does work but isn’t great. My application is going to apply migrations every time it starts – not very performant. I don’t like it.

Deploying into Docker with migrations – what does work

The resolution is a combination of the failed attempts. The principle is

  1. Provide a separate utility that can run migrations
  2. deploy this into the docker application container into it’s own folder
  3. run it after docker-compose
  4. wrap it up in a PowerShell script.

Ef Migration Utility

This is a simple console app that references the API. The app is

class Program
{
	static void Main(string[] args)
	{
		Console.WriteLine("Applying migrations");
		var webHost = new WebHostBuilder()
			.UseContentRoot(Directory.GetCurrentDirectory())
			.UseStartup<ConsoleStartup>()
			.Build();

		using (var context = (DatabaseContext) webHost.Services.GetService(typeof(DatabaseContext)))
		{
			context.Database.Migrate();
		}
		Console.WriteLine("Done");
	}
}

and the Startup class is a stripped down version of the API start up

public class ConsoleStartup
{
	public ConsoleStartup()
	{
		var builder = new ConfigurationBuilder()
			.AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
			.AddEnvironmentVariables();
		Configuration = builder.Build();
   }

	public IConfiguration Configuration { get; }

	public void ConfigureServices(IServiceCollection services)
	{
		services.AddDbContext<DatabaseContext>(options =>
		{
			options.UseMySql(Configuration.GetConnectionString("LearningAnalyticsAPIContext"));

		});
	}

	public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
	{
   
	}
}

I just need the Startup to read the app.config and get the database context up which this does. The console app references the API so it can use the API’s config files so I don’t have to double key the config into the console app.

DockerFile amends

The DockerFile file needs to be amended to deploy the migrations application into a separate folder on the app container file system. It becomes

FROM mcr.microsoft.com/dotnet/core/aspnet:3.1-nanoserver-1903 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443

FROM mcr.microsoft.com/dotnet/core/sdk:3.1-nanoserver-1903 AS build
WORKDIR /src
COPY ["LearningAnalytics.API/LearningAnalytics.API.csproj", "LearningAnalytics.API/"]
RUN dotnet restore "LearningAnalytics.API/LearningAnalytics.API.csproj"
COPY . .
WORKDIR "/src/LearningAnalytics.API"
RUN dotnet build "LearningAnalytics.API.csproj" -c Release -o /app/build

FROM mcr.microsoft.com/dotnet/core/sdk:3.1-nanoserver-1903 AS migration
WORKDIR /src
COPY . .
RUN dotnet restore "LearningAnalytics.Migration/LearningAnalytics.Migration.csproj"
COPY . .
WORKDIR "/src/LearningAnalytics.Migration"
RUN dotnet build "LearningAnalytics.Migration.csproj" -c Release -o /app/migration

FROM build AS publish
RUN dotnet publish "LearningAnalytics.API.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /migration
COPY --from=migration /app/migration .

WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "LearningAnalytics.API.dll"]

the relevant part is

FROM mcr.microsoft.com/dotnet/core/sdk:3.1-nanoserver-1903 AS migration
WORKDIR /src
COPY . .
RUN dotnet restore "LearningAnalytics.Migration/LearningAnalytics.Migration.csproj"
COPY . .
WORKDIR "/src/LearningAnalytics.Migration"
RUN dotnet build "LearningAnalytics.Migration.csproj" -c Release -o /app/migration

which builds out the migration application and …

FROM base AS final
WORKDIR /migration
COPY --from=migration /app/migration .

which copies it into a folder on the published container called migrations

Glue it together with PowerShell

Once the containers are brought up with docker-compose then it’s straightforward to use an interactive shell to navigate to the LearningAnalytics.Migration.exe application and run it. That will initialise the database. A better solution is to wrap it all up in a simple PowerShell script e.g.

docker-compose up -d
docker exec learninganalytics_app_1 c:\migration\LearningAnalytics.Migration.exe

and run that. The container comes up and the database is populated with the correct schema and reference data via EF migrations. The API now works correctly.

Entity Framework 6

The above is all for Entity Framework Core. Entity Framework 6 introduced the Migrate.exe tool . This can apply EF migrations without the source code which was the major stumbling block for EF Core. Armed with this then you could copy this up to the container and perform the migrations via something like

docker exec learninganalytics_app_1 Migration.exe

Do Migrations suck though?

This person thinks so. Certainly the inability to run them on compiled code is a huge drag. Whenever I write a production application then I prefer to just write the SQL out for the schema and apply it with some PowerShell. It’s not that hard. I like to use migrations for personal projects but there must be a reason that I’m not using them when I get paid to write code. Do I secretly think that they suck just a little?

Demo code

As ever, demo code is on my git hub site

https://github.com/timbrownls20/Learning-Analytics/tree/master/LearningAnalytics/LearningAnalytics.Migration
is the migration app

https://github.com/timbrownls20/Learning-Analytics/blob/master/LearningAnalytics/LearningAnalytics.API/DockerfileMigrations
the DockerFile

https://github.com/timbrownls20/Learning-Analytics/tree/master/LearningAnalytics
for the docker-compose.yml file and the simple PowerShell that glues it together

Useful links


This Stack Overflow question was the starting point for a lot of this and this answer particularly has a good discussion and some other options on how to achieve this – none of them are massively satisfactory. I felt something like what I’ve done was about the best.

https://docs.docker.com/compose/startup-order/
discusses why you can’t rely on the depends_on directive to make the database available to the application when you are bringing up the containers. It has more possibilities to circumvent this, such as wait-for-it. I’m certainly going to look at these but they do seem scoped to Linux rather than Windows so I’d have to change around the docker files for that. Also they wouldn’t help with Entity Framework 6 or earlier.

Leave a Reply

Your email address will not be published. Required fields are marked *