[BUG] Microsoft.Network/networkManagers@2024-05-01 cannot be deleted after IPAM pool is removed

[teowa]

API Spec link

https://github.com/Azure/azure-rest-api-specs/blob/fdcf40595e792db3a048098504a568edc113b881/specification/network/resource-manager/Microsoft.Network/stable/2024-05-01/networkManager.json

API Spec version

2024-05-01

Describe the bug

Create a network manager and an IPAM pool within it. Then, delete the IPAM pool. However, when attempting to delete the network manager, the API returns an error indicating that the IPAM pool is blocking the deletion.:

unexpected status 409 (409 Conflict) with error: CannotDeleteResource: Cannot delete resource while nested resources exist. Some existing nested resource IDs include: 'Microsoft.Network/networkManagers/acctest-nm-241127074217114551/ipamPools/acctest-ipampool-241127074217114551'. Please delete all nested resources before deleting this resource.

        deleting Network Manager (Subscription:
        "<redacted>"
        Resource Group Name: "acctestRG-network-manager-ipampool-241127074217114551"
        Network Manager Name: "acctest-nm-241127074217114551"): performing Delete:
        unexpected status 409 (409 Conflict) with error: CannotDeleteResource: Cannot
        delete resource while nested resources exist. Some existing nested resource
        IDs include:
        'Microsoft.Network/networkManagers/acctest-nm-241127074217114551/ipamPools/acctest-ipampool-241127074217114551'.
        Please delete all nested resources before deleting this resource.

Expected behavior

The network manager should be deleted successfully

Actual behavior

The network manager cannot be deleted, even if the IPAM pool has been removed. This might be due to the server-side cache not refreshing. Waiting for more than about 10 seconds allow the deletion to succeed.

Reproduction Steps

1. create a network manager and an IPAM pool within it

PUT /subscriptions/<redacted>/resourceGroups/acctestRG-network-manager-ipampool-241127074104109592/providers/Microsoft.Network/networkManagers/acctest-nm-241127074104109592?api-version=2024-05-01 HTTP/1.1
Host: management.azure.com
X-Ms-Correlation-Request-Id: 39eae2cb-7525-ab81-3842-4e2dbc8a7fbd

{"location":"westeurope","name":"acctest-nm-241127074104109592","properties":{"description":"","networkManagerScopeAccesses":["Connectivity"],"networkManagerScopes":{"managementGroups":[],"subscriptions":["/subscriptions/<redacted>"]}},"tags":{}}

PUT /subscriptions/<redacted>/resourceGroups/acctestRG-network-manager-ipampool-241127074104109592/providers/Microsoft.Network/networkManagers/acctest-nm-241127074104109592/ipamPools/acctest-ipampool-241127074104109592?api-version=2024-05-01 HTTP/1.1
Host: management.azure.com
X-Ms-Correlation-Request-Id: 39eae2cb-7525-ab81-3842-4e2dbc8a7fbd

{"location":"westeurope","name":"acctest-ipampool-241127074104109592","properties":{"addressPrefixes":["10.0.0.0/27"],"description":"","displayName":"ipampool1","parentPoolName":""},"tags":{}}

2. delete IPAM pool and check it is not found

DELETE /subscriptions/<redacted>/resourceGroups/acctestRG-network-manager-ipampool-241127074104109592/providers/Microsoft.Network/networkManagers/acctest-nm-241127074104109592/ipamPools/acctest-ipampool-241127074104109592?api-version=2024-05-01 HTTP/1.1
Host: management.azure.com
X-Ms-Correlation-Request-Id: 39eae2cb-7525-ab81-3842-4e2dbc8a7fbd

2024/11/27 07:42:12 [DEBUG] GET https://management.azure.com/subscriptions/<redacted>/resourceGroups/acctestRG-network-manager-ipampool-241127074104109592/providers/Microsoft.Network/networkManagers/acctest-nm-241127074104109592/ipamPools/acctest-ipampool-241127074104109592?api-version=2024-05-01
2024/11/27 07:42:16 [DEBUG] AzureRM Response for https://management.azure.com/subscriptions/<redacted>/resourceGroups/acctestRG-network-manager-ipampool-241127074104109592/providers/Microsoft.Network/networkManagers/acctest-nm-241127074104109592/ipamPools/acctest-ipampool-241127074104109592?api-version=2024-05-01: 
HTTP/2.0 404 Not Found
Content-Length: 346
Cache-Control: no-cache
Content-Type: application/json; charset=utf-8
Date: Wed, 27 Nov 2024 07:42:15 GMT
Expires: -1
Mise-Correlation-Id: 16697576-df33-472f-ae4b-22e6b1c31cce
Pragma: no-cache
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Cache: CONFIG_NOCACHE
X-Content-Type-Options: nosniff
X-Ms-Correlation-Request-Id: 39eae2cb-7525-ab81-3842-4e2dbc8a7fbd
X-Ms-Ratelimit-Remaining-Subscription-Global-Reads: 3749
X-Ms-Ratelimit-Remaining-Subscription-Reads: 249
X-Ms-Request-Id: 5d1e249a-eb14-4b74-8e8e-2462508e37fa
X-Ms-Routing-Request-Id: JAPANWEST:20241127T074216Z:5d1e249a-eb14-4b74-8e8e-2462508e37fa
X-Msedge-Ref: Ref A: 8DB2C86310C3400995A21A8FE6017E37 Ref B: TYO201151006023 Ref C: 2024-11-27T07:42:13Z

{"code":"NotFound","message":"Ipam pool '/subscriptions/<redacted>/resourceGroups/acctestRG-network-manager-ipampool-241127074104109592/providers/Microsoft.Network/networkManagers/acctest-nm-241127074104109592/ipamPools/acctest-ipampool-241127074104109592' not found.","target":null,"details":null,"additionalInfo":null}

3. delete network manager returns error:

2024/11/27 07:42:16 [DEBUG] DELETE https://management.azure.com/subscriptions/<redacted>/resourceGroups/acctestRG-network-manager-ipampool-241127074104109592/providers/Microsoft.Network/networkManagers/acctest-nm-241127074104109592?api-version=2024-05-01&force=true
2024/11/27 07:42:17 [DEBUG] AzureRM Response for https://management.azure.com/subscriptions/<redacted>/resourceGroups/acctestRG-network-manager-ipampool-241127074104109592/providers/Microsoft.Network/networkManagers/acctest-nm-241127074104109592?api-version=2024-05-01&force=true: 
HTTP/2.0 409 Conflict
Content-Length: 328
Cache-Control: no-cache
Content-Type: application/json; charset=utf-8
Date: Wed, 27 Nov 2024 07:42:16 GMT
Expires: -1
Pragma: no-cache
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Cache: CONFIG_NOCACHE
X-Content-Type-Options: nosniff
X-Ms-Correlation-Request-Id: 39eae2cb-7525-ab81-3842-4e2dbc8a7fbd
X-Ms-Failure-Cause: gateway
X-Ms-Request-Id: edd09d09-beb9-45ae-8931-817d269c7371
X-Ms-Routing-Request-Id: JAPANEAST:20241127T074217Z:edd09d09-beb9-45ae-8931-817d269c7371
X-Msedge-Ref: Ref A: 125C826B9DB64833A8EB93524B4275FC Ref B: TYO201100117023 Ref C: 2024-11-27T07:42:16Z

{"error":{"code":"CannotDeleteResource","message":"Cannot delete resource while nested resources exist. Some existing nested resource IDs include: 'Microsoft.Network/networkManagers/acctest-nm-241127074104109592/ipamPools/acctest-ipampool-241127074104109592'. Please delete all nested resources before deleting this resource."}}

Environment

User-Agent: HashiCorp/go-azure-sdk (Go-http-Client/1.1 networkmanagers/2024-05-01) HashiCorp Terraform/1.9.4 (+https://www.terraform.io) terraform-provider-azurerm/acc pid-222c6c49-1b0a-5959-a213-6608f9eb8820

[v-jiaodi]

@kamboj-prjwl Please help take a look, thanks.