[Docker/Registry] Troubleshooting Image Pull Failures in Self-Built Container Management Tool that Uses Docker Compose SDK

The Problem

I’m building a container management tool using the Docker Compose SDK (link) and the Docker CLI library (link). One feature is the ability to deploy Docker Compose YAML files that reference images from private registries.

To handle authentication, I added private registry credentials into my tool and injected them into the Docker client using the authConfig field during Compose service initialization. I expected Docker Compose to use these credentials when pulling images.

The first test using private registry A worked. But when I switched to registry B, it failed. After debugging, I found that registry A was already configured in my local ~/.docker/config.json file. Docker had access to its credentials without relying on what my tool provided.

Here’s what the config.json looked like:

{
  "auths": {
    "registryA": {
        ...
    }
  },
  "credsStore": "desktop.exe"
}

This explained why registry A worked: Docker was using credentials from the local config file, not the injected ones.

To test further, I removed the stored registry credentials and tried again. This time the deployment failed on my Windows (WSL) environment. However, running the exact same YAML file on a clean Linux machine with no private registry credentials configured worked perfectly.

So the issue was environment-specific.

What’s Happening Behind the Scenes

My tool uses compose.Up(...) to deploy the YAML file. Internally, this calls:

cli.Client().ImagePull(...)

This method belongs to the Docker Engine SDK and needs credentials to pull the image. It retrieves them from the local Docker config, typically ~/.docker/config.json.

If this config file includes:

{ "credsStore": "desktop" }

or

{ "credHelpers": { "ghcr.io": "desktop" } }

Docker will not use the auths block. Instead, it delegates to the native credential helper (e.g., docker-credential-desktop.exe).

Even if you inject credentials into auths in the code, they are ignored when a credential helper is configured.

Why Compose SDK Follows This Path

When the Compose SDK checks for required images, it uses:

cli.ImagePull(...)

This triggers:

• Compose SDK
• Docker SDK
• Docker CLI config loader
• cli/config/credentials/native_store.go
• docker-credential-desktop.exe get

So if Docker is set up to use a native credential helper, Compose will always rely on it and not on any credentials you inject programmatically.

What Triggers the Native Credential Store

Docker uses the native credential helper when config.json includes:

{
  "credsStore": "desktop"
}

or

{
  "credHelpers": {
    "ghcr.io": "desktop"
  }
}

When either is present, Docker ignores the auths block and asks the credential helper instead.

How to Prevent This

If you want the Compose SDK to use the credentials your tool provides (inside auths), you must:

• Remove the credsStore key
• Remove the credHelpers key

That way, Docker will fall back to using the auths block.

My Approach

To avoid conflicts and ensure a clean environment, I removed credsStore from the local config.json. This made sure the Docker client used only the credentials injected by my tool.

As an improvement, the tool could show a warning if credsStore or credHelpers is detected, informing the user that those settings might override the injected credentials.

With a clean config.json, the process works like this:

1. My tool injects the private registry credentials into the Docker client.
2. The Docker client is passed to the Compose service.
3. Compose pulls the image from the private registry using the injected credentials.

Summary

• Docker resolves credentials in this order:

• On Docker Desktop (Windows/macOS), a global credsStore (e.g., desktop.exe/osxkeychain) can override injected inline auths and cause private-registry pulls to fail.