Introduction
Recently, I encountered an interesting issue while accessing Azure Container Registry (ACR) using Azure CLI.
The registry was fully operational:
- Azure login worked
- DNS resolution worked
- HTTPS connectivity worked
- Docker login worked
- Registry was healthy
However, Azure CLI commands such as:
az acr login
az acr repository list
were consistently failing with SSL certificate validation errors. The failure was ultimately isolated to SSL certificate trust validation performed by Azure CLI’s Python runtime.
This article walks through the complete troubleshooting journey, explains the underlying theory, discusses certificate trust chains, and demonstrates how to identify and resolve such issues.
| Component | Value |
|---|---|
| ACR Name | contosoacr |
| Login Server | contosoacr.azurecr.io |
| Client OS | Windows 11 |
| Azure CLI | 2.86.0 |
| Docker | 29.x |
Understanding How Azure Container Registry Authentication Works
Before diving into troubleshooting, it is important to understand the authentication flow, what actually happens when Azure CLI connects to Azure Container Registry (ACR) . or we can say Many engineers assume that az acr login simply logs into the registry. In reality, multiple Azure services, authentication systems, and security validations are involved before the login succeeds.
Understanding this workflow helps pinpoint exactly where a failure occurs and significantly reduces troubleshooting time.
User
│
│ az login
▼
Azure AD
│
▼
Azure Resource Manager
│
▼
Azure Container Registry
│
▼
Docker Registry API (/v2/)
When running:
az acr login --name contosoacr
Step 1 – Azure AD Authentication
The process begins with Azure authentication.
az login
This command authenticates the user against Microsoft Entra ID (formerly Azure Active Directory).
During this stage Azure CLI:
- Validates user credentials
- Obtains OAuth access tokens
- Stores authentication context locally
- Identifies available subscriptions and tenants
Successful authentication allows Azure CLI to act on behalf of the authenticated user.
Potential Failures
Common issues at this stage include:
- Expired credentials
- MFA challenges
- Conditional Access Policies
- Invalid tenant selection
- Expired access tokens
Typical errors:
Authentication failed
AADSTS errors
Token acquisition failed
Step 2 – Azure Resource Manager Discovery
Once authenticated, Azure CLI needs to locate the registry.
It communicates with Azure Resource Manager (ARM) to retrieve metadata about the registry.
az acr show --name contosoacr
Behind the scenes Azure CLI retrieves:
- Registry name
- Resource ID
- Login server URL
- Subscription information
- Registry configuration
Example:
{
"name": "contosoacr",
"loginServer": "contosoacr.azurecr.io"
}
At this point Azure CLI knows exactly where the registry resides.
Potential Failures
Possible issues include:
- Incorrect registry name
- Wrong subscription selected
- Missing RBAC permissions
- Registry deleted or unavailable
Typical errors:
ResourceNotFound
AuthorizationFailed
SubscriptionNotFound
Step 3 – DNS Resolution
Before connecting to the registry endpoint, the hostname must be translated into an IP address.
Azure CLI attempts to resolve:
contosoacr.azurecr.io
through DNS.
Example:
nslookup contosoacr.azurecr.io
Result:
20.x.x.x
Without successful DNS resolution, no network connection can be established.
Potential Failures
- Corporate DNS issues
- Split-brain DNS configuration
- Private endpoint misconfiguration
- DNS forwarding issues
Typical errors:
Name or service not known
DNS lookup failed
Step 4 – Network Connectivity
After DNS resolution succeeds, Azure CLI attempts to establish network connectivity to the registry endpoint.
The registry listens on:
TCP 443
for HTTPS traffic.
Example validation:
Test-NetConnection contosoacr.azurecr.io -Port 443
Successful connectivity confirms:
- Routing exists
- Firewall rules permit traffic
- Security groups allow access
- Proxy configuration is functioning
Potential Failures
Common causes include:
- Firewall restrictions
- Proxy misconfiguration
- VPN routing issues
- Network segmentation
Typical errors:
Connection timeout
Connection refused
Unable to connect
Step 5 – TLS Handshake
Once the TCP connection is established, a TLS handshake begins.
This is where secure communication is negotiated between:
Azure CLI
│
▼
Azure Container Registry
During the handshake:
- Encryption algorithms are negotiated
- Certificates are exchanged
- Session keys are generated
- Secure communication channel is established
This step ensures data cannot be intercepted or modified during transit.
Potential Failures
- TLS version mismatch
- Certificate problems
- SSL inspection devices
- Expired certificates
Typical errors:
TLS handshake failed
SSL connection failed
Step 6 – Certificate Validation
This is one of the most critical stages.
After receiving the registry’s certificate, Azure CLI must validate the certificate chain.
The chain typically looks like:
Server Certificate
│
▼
Intermediate CA
│
▼
Root CA
Azure CLI verifies:
- Certificate is not expired
- Certificate matches hostname
- Certificate chain is complete
- Root CA is trusted
Only after all validations pass will the connection continue.
Potential Failures
This is where our issue occurred.
Common causes include:
- Missing intermediate certificates
- Corporate SSL inspection
- Untrusted corporate root CA
- Outdated CA bundles
- Corrupted certificate stores
Typical errors:
certificate verify failed
unable to get local issuer certificate
CERTIFICATE_VERIFY_FAILED
Step 7 – OAuth Token Exchange
After secure communication is established, Azure CLI requests an ACR access token.
Flow:
Azure CLI
│
▼
Microsoft Entra ID
│
▼
ACR Access Token
The token grants temporary access to registry resources.
The token contains permissions based on:
- User identity
- Azure RBAC roles
- ACR repository permissions
Potential Failures
- Insufficient permissions
- Expired tokens
- RBAC misconfiguration
Typical errors:
Unauthorized
Forbidden
Authentication required
Step 8 – Docker Authentication
Finally Azure CLI passes credentials to Docker.
Azure CLI
│
▼
Docker Credential Store
│
▼
Azure Container Registry
This enables Docker operations such as:
docker pull
docker push
docker build
without requiring additional credentials.
When everything succeeds:
az acr login --name contosoacr
returns:
Login Succeeded
Where Can the Process Fail?
Every successful ACR login depends on multiple layers working together.
Azure Authentication
│
▼
Azure Authorization
│
▼
DNS Resolution
│
▼
Network Connectivity
│
▼
TLS Handshake
│
▼
Certificate Validation
│
▼
OAuth Token Exchange
│
▼
Docker Authentication
A failure at any single stage can prevent access to the registry. This is why effective troubleshooting requires validating each layer independently rather than assuming the issue is network-related.
In our case, every stage succeeded until Certificate Validation, where Azure CLI’s Python runtime was unable to trust the certificate chain, causing the entire authentication flow to fail.
Initial Error
A user attempts to access Azure Container Registry using Azure CLI.
User executes:
az acr login --name contosoacr
Output:
Could not connect to the registry login server
'https://contosoacr.azurecr.io/v2/'
Try running:
az acr check-health
At first glance this appears to be:
- Network issue
- DNS issue
- Firewall issue
- ACR outage
But none of those turned out to be true.
Step 1 – Verify Azure Authentication
Command:
az login
Verify subscription:
az account show
Expected:
{
"name": "Contoso-Production",
"state": "Enabled"
}
Result:
Successful
Conclusion:
Azure authentication is working.
Step 2 – Verify ACR Exists
Command:
az acr show --name contosoacr
Expected:
{
"name": "contosoacr",
"provisioningState": "Succeeded"
}
Result:
Successful
Conclusion:
Registry exists and Azure Resource Manager can access it.
Step 3 – Verify DNS Resolution
Command:
nslookup contosoacr.azurecr.io
Expected:
Name: contosoacr.azurecr.io
Address: 20.x.x.x
Result:
Successful
Conclusion:
DNS resolution is working.
Step 4 – Verify Network Connectivity
Command:
Test-NetConnection contosoacr.azurecr.io -Port 443
Output:
TcpTestSucceeded : True
Result:
Successful
Conclusion:
TCP connectivity to HTTPS endpoint is working.
Step 5 – Verify ACR Endpoint Reachability
Command:
curl https://contosoacr.azurecr.io/v2/
Response:
{
"errors": [
{
"code": "UNAUTHORIZED",
"message": "authentication required"
}
]
}
Many engineers mistakenly think this is an error.
It is actually expected.
Why?
Because:
/v2/
is the Docker Registry API endpoint and requires authentication.
Result:
Successful
Conclusion:
HTTPS handshake completed. The registry endpoint is reachable and responding correctly.
Step 6 – Docker Validation
Command:
docker login contosoacr.azurecr.io
Result:
Login Succeeded
Conclusion:
Registry accepts credentials.
Registry service healthy.
Step 7 – Run ACR Health Check
The az acr check-health command is one of the most valuable diagnostics available for ACR troubleshooting. It validates Docker availability, DNS resolution, registry reachability, and SSL connectivity, making it an excellent first step when diagnosing registry access issues.
Reference: Azure Container Registry Health Check Documentation
Command:
az acr check-health --name contosoacr --yes
Output:
CONNECTIVITY_SSL_ERROR
Unable to establish a secure connection
to the registry.
This was the first major clue.
The issue was no longer:
- DNS
- Firewall
- Routing
Instead it pointed toward SSL/TLS. This is the first indication of certificate-related issues.
Step 8 – Enable Debug Logging
Command:
az acr login \
--name contosoacr \
--debug
Important section:
SSLCertVerificationError:
certificate verify failed:
unable to get local issuer certificate
This message revealed the actual root cause.
Root Cause Analysis
The registry itself was healthy.
The network path was healthy.
The authentication flow was healthy.
The failure occurred during certificate validation.
Specifically:
Azure CLI
│
▼
Python Requests Library
│
▼
Certificate Chain Validation
│
▼
FAILED
Understanding SSL Certificate Chains
When a client connects to:
https://contosoacr.azurecr.io
the server presents:
Server Certificate
│
▼
Intermediate CA
│
▼
Root CA
The client must trust the entire chain.
If any certificate in the chain is missing or untrusted:
certificate verify failed
unable to get local issuer certificate
will occur.
Why Docker Worked But Azure CLI Failed
This confused many engineers.
Docker login succeeded:
docker login contosoacr.azurecr.io
But Azure CLI failed.
Why?
Different applications use different trust stores.
Docker
Typically trusts:
- Windows Certificate Store
- Docker Certificate Store
Azure CLI
Uses:
Python Requests
urllib3
OpenSSL
and may rely on a separate CA bundle.
Therefore:
Docker → Success
Azure CLI → Failure
is entirely possible.
Common Enterprise Root Causes
1. SSL Inspection Proxy
Examples:
- Zscaler
- Netskope
- Blue Coat
- Palo Alto
- Forcepoint
These solutions:
Client
│
▼
Corporate Proxy
│
▼
Azure Container Registry
The proxy replaces Microsoft’s certificate with an internal corporate certificate.
Windows may trust it.
Python may not.
Result:
certificate verify failed : unable to get local issuer certificate
What needs to be checked from certificate side
Review:
Certificate Path
Certificate Issuer
Certificate Subject
Verify whether:
Corporate Root CA
Corporate Intermediate CA
are trusted.
2. Missing Intermediate Certificate
Certificate chain is incomplete.
Client cannot build trust chain.
Symptoms:
Docker Works
Browser Works
Azure CLI Fails
Certificate Team Actions
Validate:
Server Certificate
Intermediate Certificate
Root Certificate
Ensure complete chain is available.
3. Outdated CA Bundle
Azure CLI installation contains outdated certificates.
Desktop Team Actions
Open:
certlm.msc
Check:
Trusted Root Certification Authorities
Verify:
Corporate Root CA
exists.
4. Corrupted Certificate Store
Local workstation certificate store is inconsistent.
Desktop Team Actions
Upgrade Azure CLI:
winget upgrade Microsoft.AzureCLI
or reinstall Azure CLI.
How To Identify SSL Inspection
Check proxy configuration:
netsh winhttp show proxy
Check environment variables:
set HTTP_PROXY
set HTTPS_PROXY
Look for:
Zscaler
Netskope
Bluecoat
Corporate Gateway
Temporary Workaround
If Docker authentication works:
docker login contosoacr.azurecr.io
You can still push images.
Example:
docker build -t webapp:v1 .
Tag:
docker tag webapp:v1 \
contosoacr.azurecr.io/webapp:v1
Push:
docker push \
contosoacr.azurecr.io/webapp:v1
This bypasses Azure CLI authentication.
Permanent Resolution
Resolution Path 1 – Verify Certificate Chain
Certificate Team should inspect:
https://contosoacr.azurecr.io
Validate:
- Server Certificate
- Intermediate CA
- Root CA
Tools:
OpenSSL
Browser Certificate Viewer
SSL Labs
Enterprise PKI Tools
Resolution Path 2 – Validate SSL Inspection
Security Team should verify:
Is SSL Inspection enabled?
If yes:
Which certificate is being injected?
Common examples:
Zscaler Root CA
Netskope Root CA
Corporate Inspection CA
Resolution Path 3 – Import Missing CA
If a corporate CA is identified:
Import into:
Local Computer
└ Trusted Root Certification Authorities
and
Intermediate Certification Authorities
if required.
Resolution Path 4 – Python Trust Configuration
If Azure CLI still fails:
Locate corporate CA certificate.
Configure:
set REQUESTS_CA_BUNDLE=C:\certs\corp-root.pem
or
set SSL_CERT_FILE=C:\certs\corp-root.pem
Retest:
az acr login --name contosoacr
Troubleshooting Checklist
| Validation | Result |
|---|---|
| az login | Passed |
| az acr show | Passed |
| DNS Lookup | Passed |
| Port 443 Connectivity | Passed |
| curl /v2/ | Passed |
| Docker Login | Passed |
| ACR Health Check | Failed |
| Azure CLI Login | Failed |
| SSL Validation | Failed |
Validation After Fix
Run:
az acr check-health --name contosoacr --yes
Expected:
Docker daemon status: available
Docker pull test: OK
DNS lookup: OK
Registry HTTPS connectivity: OK
Then:
az acr login --name contosoacr
Expected:
Login Succeeded
Verify:
az acr repository list --name contosoacr
Expected:
repository1
repository2
repository3
Key Takeaways
When troubleshooting Azure Container Registry issues:
Do not assume every “Could not connect to registry” error is a network problem.
Always validate:
- Azure authentication
- DNS
- TCP connectivity
- HTTPS endpoint reachability
- Docker login
- SSL certificate chain
- Azure CLI debug logs
The single most valuable command during this investigation was:
az acr login --name contosoacr --debug
which exposed the underlying SSL certificate validation failure.
In our case, the registry, networking, firewall rules, and permissions were all functioning correctly. The root cause was ultimately isolated to certificate trust validation performed by Azure CLI’s Python runtime.
Understanding the distinction between connectivity issues and certificate trust issues can save many hours of troubleshooting in enterprise environments.
Leave a Reply