【Azure API 管理】APIM Self-Host Gateway 自建本地环境中的网关数量超过10个且它们的出口IP为同一个时出现的429错误

问题描述

Azure API Management服务支持使用自建网关来实现API服务，这样APIM的所有请求都可以走在自管理的企业内网中。流量都更加安全。当根据官网文档搭建后，如果在本地环境中部署超过10个Self-host gateway，并且这些Self-host gateway的主机出口IP地址（公网IP)都是同一个的时候，就会遇见429错误： [HeartbeatError], exception: System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 ().

更多的错误消息

This software is licensed to you as part of your or your company's subscription license for Microsoft Azure Services.

You may only use the software with Microsoft Azure Services and subject to the terms and conditions of the agreement under which you obtained Microsoft Azure Services.

If you do not have an active subscription license for Microsoft Azure Services, you may not use the software.

See Microsoft Azure Legal Information: https://azure.microsoft.com/en-us/support/legal/

__ __ ___ __ __ ___ ___ ___

/\ / | | |__) |__ /\ |__) | |\/| /\ |\ | /\ / _` |__ |\/| |__ |\ | |

/~~\ /_ \__/ | \ |___ /~~\ | | | | /~~\ | \| /~~\ \__> |___ | | |___ | \| |

____ _ __ _ _ _ _ ____ _

/ ___| ___ | | / _| | | | | ___ ___ | |_ ___ __| | / ___| __ _ | |_ ___ __ __ __ _ _ _

\___ \ / _ \ | | | |_ _____ | |_| | / _ \ / __| | __| / _ \ / _` | | | _ / _` | | __| / _ \ \ \ /\ / / / _` | | | | |

___) | | __/ | | | _| |_____| | _ | | (_) | \__ \ | |_ | __/ | (_| | | |_| | | (_| | | |_ | __/ \ V V / | (_| | | |_| |

|____/ \___| |_| |_| |_| |_| \___/ |___/ \__| \___| \__,_| \____| \__,_| \__| \___| \_/\_/ \__,_| \__, |

|___/

.%*

/##(

#####.

,%%%#%%/

#%%%%%%%%

(((#####((((#/

((((##(((//////((

((((##(///*******//.

/(((%#(//**,,..,,,*//

%%%%%%%%############(/############%%%%%%%%%%%%%%%%%((/**,.. ..,,**

.**(/%/%%%%%%((/**,... ..,***

(((%#((/**,,,,,,**/.

(((%#((//****,,//

((%##((((///(*

,%%####

%%%%%,

*%%#%

(###

##(

,#/

_ _

| |__ ___ ___ _ __ | |__ ___ ___ _ __

| '_ \ / _ \ / _ \ | '_ \ _____ | '_ \ / _ \ / _ \ | '_ \

| |_) | | __/ | __/ | |_) | |_____| | |_) | | __/ | __/ | |_) |

|_.__/ \___| \___| | .__/ |_.__/ \___| \___| | .__/

|_| |_|

version: 1.2.5.0

commit: 2781a2bc38328bb1f9b425342e847a683105b7fa

[Warn] 2021-12-7T10:26:33.418 [LoadBackupLocationNotFound], message: /apim/config, source: BackupMetadataProvider

[Info] 2021-12-7T10:26:33.471 [HostBootstrapperStarting]

[Info] 2021-12-7T10:26:33.471 [ComponentStarting], message: EventListenerComponent

[Info] 2021-12-7T10:26:33.471 [ComponentStarted], message: EventListenerComponent

[Info] 2021-12-7T10:26:33.471 [ComponentStarting], message: EndPointEventListenerHost

[Info] 2021-12-7T10:26:33.473 [ComponentStarted], message: EndPointEventListenerHost

[Info] 2021-12-7T10:26:33.492 [LoadingConfiguration], message: https://apim-restry.management.azure-api.cn/subscriptions/6630b017-4a45-44cc-a150-5e413bf22091/resourceGroups/RG-APIM/providers/Microsoft.ApiManagement/service/apim-restry/gateways/getConfiguration?api-version=2018-06-01-preview, source: ServiceConfigurationSource

[Info] 2021-12-7T10:26:34.364 [ConfigurationUpdated], message: Updated settings: tenant.id, tenant.name, config.enc, config.enc.keys.primary, config.enc.keys.secondary, config.cert.privatekey.password, deployment.dataset.id, logs.applicationinsights.endpoint, policy.qouta.sync.table.connection, events.snapshot.initial.connection, events.snapshot.private.master.connection, events.table.connection, inspector.container.connection, deployment.instance.region, deployment.instance.gateway-id, deployment.id, deployment.name, config.mode, source: ServiceConfigurationSource

[Info] 2021-12-7T10:26:34.371 [ComponentStarting], message: ExternalConfigurationHeartbeat

[Info] 2021-12-7T10:26:34.372 [ComponentStarted], message: ExternalConfigurationHeartbeat

[Info] 2021-12-7T10:26:34.372 [ComponentStarting], message: ExternalConfigurationMonitor

[Info] 2021-12-7T10:26:34.372 [ComponentStarted], message: ExternalConfigurationMonitor

[Info] 2021-12-7T10:26:34.372 [ComponentStarting], message: P2PRateLimitSignalSenderService

[Info] 2021-12-7T10:26:34.373 [ComponentStarted], message: P2PRateLimitSignalSenderService

[Info] 2021-12-7T10:26:34.373 [ComponentStarting], message: P2PRateLimitSignalListenerService

[Info] 2021-12-7T10:26:34.374 [ComponentStarted], message: P2PRateLimitSignalListenerService

[Info] 2021-12-7T10:26:34.374 [ComponentStarting], message: StubHostableComponent

[Info] 2021-12-7T10:26:34.374 [ComponentStarted], message: StubHostableComponent

[Info] 2021-12-7T10:26:34.374 [ComponentStarting], message: StubHostableComponent

[Info] 2021-12-7T10:26:34.374 [ComponentStarted], message: StubHostableComponent

[Info] 2021-12-7T10:26:34.374 [ComponentStarting], message: NeighborDiscoveryService

[Info] 2021-12-7T10:26:34.384 [NeighborhoodChangeDetected], message: [] --> [255.255.255.255: 255.255.255.255], source: UdpHeartbeatSender

[Info] 2021-12-7T10:26:34.385 [NeighborhoodChangeDetected], message: [] --> [255.255.255.255: 255.255.255.255], source: P2PRateLimitSignalSenderService

[Info] 2021-12-7T10:26:34.385 [ComponentStarted], message: NeighborDiscoveryService

[Info] 2021-12-7T10:26:34.385 [ComponentStarting], message: DotNettyBootstrapper

[Info] 2021-12-7T10:26:34.393 [SelfAddressDiscovery], message: Rate limit is running in UDP broadcast mode. Discovering local node addresses to ignore loopback datagrams..., source: P2PRateLimitSignalListenerService

[Info] 2021-12-7T10:26:34.420 [ComponentStarted], message: DotNettyBootstrapper

[Info] 2021-12-7T10:26:34.420 [ComponentStarting], message: AspNetCoreBootstrapper

[Info] 2021-12-7T10:26:34.463 [MyAddresses], message: 127.0.0.1, 172.18.0.12, source: P2PRateLimitSignalListenerService

[Info] 2021-12-7T10:26:34.466 [SignalListenerStarting], message: 0.0.0.0:4290, source: P2PRateLimitSignalListenerService

[Info] 2021-12-7T10:26:34.470 [SignalListenerStarted], message: 0.0.0.0:4290, source: P2PRateLimitSignalListenerService

Hosting environment: Production

Content root path: /app

Now listening on: http://0.0.0.0:8080

Now listening on: https://0.0.0.0:8081

[Info] 2021-12-7T10:26:34.665 [ComponentStarted], message: AspNetCoreBootstrapper

[Info] 2021-12-7T10:26:34.665 [HostBootstrapperStarted]

[Info] 2021-12-7T10:26:34.954 [TenantStarting], message: 61af36da3aaab00001747dc7

[Info] 2021-12-7T10:26:35.501 [KubernetesIngressModeDisabled], source: KubernetesConfigurationRepositoryProvider

[Warn] 2021-12-7T10:26:35.599 [LoadBackupLocationNotFound], message: /apim/config, source: FileBackupProvider

[Info] 2021-12-7T10:26:35.901 [LegacyBackendUpdated], source: BackendService

[Info] 2021-12-7T10:26:36.314 [BootstrapperStarting], source: Bootstrapper

[Info] 2021-12-7T10:26:36.314 [ComponentStarting], message: PolicyMapBuilder, source: Bootstrapper

[Info] 2021-12-7T10:26:36.315 [ComponentStarted], message: PolicyMapBuilder, source: Bootstrapper

[Info] 2021-12-7T10:26:36.315 [ComponentStarting], message: RequestHandlerContainer, source: Bootstrapper

[Info] 2021-12-7T10:26:36.315 [ComponentStarted], message: RequestHandlerContainer, source: Bootstrapper

[Info] 2021-12-7T10:26:36.315 [ComponentStarting], message: RequestHandlerContainer, source: Bootstrapper

[Info] 2021-12-7T10:26:36.315 [ComponentStarted], message: RequestHandlerContainer, source: Bootstrapper

[Info] 2021-12-7T10:26:36.315 [ComponentStarting], message: RequestHandlerContainer, source: Bootstrapper

[Info] 2021-12-7T10:26:36.315 [ComponentStarted], message: RequestHandlerContainer, source: Bootstrapper

[Info] 2021-12-7T10:26:36.315 [ComponentStarting], message: RequestHandlerContainer, source: Bootstrapper

[Info] 2021-12-7T10:26:36.315 [ComponentStarted], message: RequestHandlerContainer, source: Bootstrapper

[Info] 2021-12-7T10:26:36.315 [ComponentStarting], message: ValidateMessageService, source: Bootstrapper

[Info] 2021-12-7T10:26:36.369 [ComponentStarted], message: ValidateMessageService, source: Bootstrapper

[Info] 2021-12-7T10:26:36.369 [ComponentStarting], message: MetricPublisher, source: Bootstrapper

[Info] 2021-12-7T10:26:36.450 [MetricPublicationStarted], source: MetricsPublisher

[Info] 2021-12-7T10:26:36.450 [ComponentStarted], message: MetricPublisher, source: Bootstrapper

[Info] 2021-12-7T10:26:36.450 [ComponentStarting], message: QuotaComponent, source: Bootstrapper

[Info] 2021-12-7T10:26:36.451 [ComponentStarted], message: QuotaComponent, source: Bootstrapper

[Info] 2021-12-7T10:26:36.451 [ComponentStarting], message: CloudMetricConnection, source: Bootstrapper

[Info] 2021-12-7T10:26:36.477 [ComponentStarted], message: CloudMetricConnection, source: Bootstrapper

[Info] 2021-12-7T10:26:36.479 [ConfigStarted], source: CompositeConfigurationRepositoryProvider

[Info] 2021-12-7T10:26:36.479 [ConfigStarted], message: Connected service is enabled. 'config.service.endpoint' is set., source: CompositeConfigurationRepositoryProvider

[Info] 2021-12-7T10:26:36.488 [ConfigInitialSyncStarted], source: ConfigurationRepositoryProvider

[Info] 2021-12-7T10:26:36.752 [CounterInitialSyncStarted], source: QuotaByKeyCounterService

[Info] 2021-12-7T10:26:36.776 [CounterInitialSyncCompleted], source: QuotaByKeyCounterService

[Info] 2021-12-7T10:26:37.170 [EventSnapshotElected], message: provider: storage, uri: https://apimstqax0aibucxyvra4cdf.blob.core.chinacloudapi.cn/gatewaysnapshotsrxwmzisgzzfiv7r/snapshot-2019-10-11.Proxy.Host.WebRole_IN_0.json.gzip, rev: 225, source: events.snapshot

[Info] 2021-12-7T10:26:37.180 [ConfigurationRetrieving], message: https://apimstqax0aibucxyvra4cdf.blob.core.chinacloudapi.cn/gatewaysnapshotsrxwmzisgzzfiv7r/snapshot-2019-10-11.Proxy.Host.WebRole_IN_0.json.gzip, source: events.snapshot.storage.private

[Info] 2021-12-7T10:26:37.266 [ConfigurationLoaded], message: https://apimstqax0aibucxyvra4cdf.blob.core.chinacloudapi.cn/gatewaysnapshotsrxwmzisgzzfiv7r/snapshot-2019-10-11.Proxy.Host.WebRole_IN_0.json.gzip, source: events.snapshot.storage.private

[Info] 2021-12-7T10:26:37.712 [ConfigurationRetrieved], message: https://apimstqax0aibucxyvra4cdf.blob.core.chinacloudapi.cn/gatewaysnapshotsrxwmzisgzzfiv7r/snapshot-2019-10-11.Proxy.Host.WebRole_IN_0.json.gzip, source: events.snapshot.storage.private

[Info] 2021-12-7T10:26:37.721 [LocalLoggerAddedToTenant], message: apim-restry.azure-api.cn, source: DefaultLocalLoggerConfigurationFilter

[Info] 2021-12-7T10:26:37.723 [LocalDiagnosticAddedtoTenant], message: apim-restry.azure-api.cn, source: DefaultTenantDiagnosticConfigurationFilter

[Warn] 2021-12-7T10:26:37.811 [Logger azuremonitor with unknown type azureMonitor detected], source: LoggerResolver

[Info] 2021-12-7T10:26:37.885 [OperationRouteTableRebuildStarted], message: echo-api;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.973 [OperationRouteTableRebuildCompleted], message: echo-api;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.981 [OperationRouteTableRebuildStarted], message: auction;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.989 [OperationRouteTableRebuildCompleted], message: auction;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.990 [OperationRouteTableRebuildStarted], message: todo-api;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.994 [OperationRouteTableRebuildCompleted], message: todo-api;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.994 [OperationRouteTableRebuildStarted], message: apim;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.994 [OperationRouteTableRebuildCompleted], message: apim;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.994 [OperationRouteTableRebuildStarted], message: api101;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.994 [OperationRouteTableRebuildCompleted], message: api101;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.994 [OperationRouteTableRebuildStarted], message: api103;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.995 [OperationRouteTableRebuildCompleted], message: api103;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.995 [OperationRouteTableRebuildStarted], message: api112;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.995 [OperationRouteTableRebuildCompleted], message: api112;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.995 [OperationRouteTableRebuildStarted], message: api114;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.995 [OperationRouteTableRebuildCompleted], message: api114;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.995 [OperationRouteTableRebuildStarted], message: apiID_self-getway;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.995 [OperationRouteTableRebuildCompleted], message: apiID_self-getway;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.995 [OperationRouteTableRebuildStarted], message: apiID_dev-nm-domestic;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.995 [OperationRouteTableRebuildCompleted], message: apiID_dev-nm-domestic;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.995 [OperationRouteTableRebuildStarted], message: apiid-dev-bj-domestic;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.995 [OperationRouteTableRebuildCompleted], message: apiid-dev-bj-domestic;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.995 [OperationRouteTableRebuildStarted], message: api-self-getway;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:37.995 [OperationRouteTableRebuildCompleted], message: api-self-getway;rev=1, source: ApiRouter

[Info] 2021-12-7T10:26:38.230 [LegacyBackendUpdated], source: BackendService

[Info] 2021-12-7T10:26:38.231 [EventSnapshotRestored], message: revision: 000000e1, datasetId: rxwmzisgzzfiv7r, source: ConfigurationRepositoryProvider

[Info] 2021-12-7T10:26:38.246 [WaitingForRemainingEvents], message: snapshot-rev: 000000e1, source: ConfigurationRepositoryProvider

[Info] 2021-12-7T10:26:38.249 [EventLoopStopped], source: TableStorageEventLoopFactory

[Info] 2021-12-7T10:26:38.249 [EventLoopStarted], source: TableStorageEventLoopFactory

[Info] 2021-12-7T10:26:38.713 [NewEventsRead], message: count: 1, source: TableStorageEventLoopFactory

[Info] 2021-12-7T10:26:38.714 [EventProcessing], message: partitionkey: apim-restry.azure-api.cn, rowkey: 000000e2, etag: W/"datetime'2021-12-07T10%3A24%3A59.458422Z'", source: TableStorageEventLoopFactory

[Info] 2021-12-7T10:26:38.729 [EntityEventProcessing], message: id: 000000e2, entity-id: /gateways/61af367b58ac0a10a44e6e85, public-entity-id: /gateways/dev-gateway-03, type: created, entity-rev: 8010, source: ConfigurationRepositoryProvider

[Info] 2021-12-7T10:26:39.138 [EntityEventProcessed], message: id: 000000e2, entity-id: /gateways/61af367b58ac0a10a44e6e85, public-entity-id: /gateways/dev-gateway-03, type: created, entity-rev: 8010, source: ConfigurationRepositoryProvider

[Info] 2021-12-7T10:26:39.138 [EventProcessed], message: partitionkey: apim-restry.azure-api.cn, rowkey: 000000e2, etag: W/"datetime'2021-12-07T10%3A24%3A59.458422Z'", source: TableStorageEventLoopFactory

[Info] 2021-12-7T10:26:39.139 [EventsSuccessfullyRestored], source: ConfigurationRepositoryProvider

[Info] 2021-12-7T10:26:39.142 [ConfigInitialSyncCompleted], source: ConfigurationRepositoryProvider

[Info] 2021-12-7T10:26:39.143 [BootstrapperStarted], source: Bootstrapper

[Info] 2021-12-7T10:26:39.144 [TenantStarted], message: 61af36da3aaab00001747dc7 00:00:04.4720580

[Error]2021-12-7T10:27:33.543 [HeartbeatError], exception: System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 ().

at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()

at Gateway.Host.AspNetCore.ExternalConfiguration.ExternalConfigurationHeartbeat.g__Send|11_1(CancellationToken cancellation) in D:\a\1\s\Proxy\Gateway.Host.AspNetCore\ExternalConfiguration\ExternalConfigurationHeartbeat.cs:line 105, source: ConfigService

[Error]2021-12-7T10:28:33.550 [HeartbeatError], exception: System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 ().

at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()

[Error]2021-12-7T10:29:33.562 [HeartbeatError], exception: System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 ().

at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()

[Error]2021-12-7T10:30:33.678 [HeartbeatError], exception: System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 ().

at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()

[Error]2021-12-7T10:31:33.691 [HeartbeatError], exception: System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 ().

at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()

[Error]2021-12-7T10:32:33.702 [HeartbeatError], exception: System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 ().

at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()

[Error]2021-12-7T10:33:33.711 [HeartbeatError], exception: System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 ().

at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()

[Error]2021-12-7T10:34:33.719 [HeartbeatError], exception: System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 ().

at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()

[Error]2021-12-7T10:35:33.725 [HeartbeatError], exception: System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 ().

at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()

[Info] 2021-12-7T10:35:49.546 [NewEventsRead], message: count: 1, source: TableStorageEventLoopFactory

[Info] 2021-12-7T10:35:49.546 [EventProcessing], message: partitionkey: apim-restry.azure-api.cn, rowkey: 000000e3, etag: W/"datetime'2021-12-07T10%3A35%3A43.601638Z'", source: TableStorageEventLoopFactory

[Info] 2021-12-7T10:35:49.546 [EntityEventProcessing], message: id: 000000e3, entity-id: /gateways/61af38ff58ac0a10a44e6f13, public-entity-id: /gateways/dev-gateway-04, type: created, entity-rev: 8015, source: ConfigurationRepositoryProvider

[Info] 2021-12-7T10:35:49.547 [EntityEventProcessed], message: id: 000000e3, entity-id: /gateways/61af38ff58ac0a10a44e6f13, public-entity-id: /gateways/dev-gateway-04, type: created, entity-rev: 8015, source: ConfigurationRepositoryProvider

[Info] 2021-12-7T10:35:49.547 [EventProcessed], message: partitionkey: apim-restry.azure-api.cn, rowkey: 000000e3, etag: W/"datetime'2021-12-07T10%3A35%3A43.601638Z'", source: TableStorageEventLoopFactory

[Error]2021-12-7T10:36:33.730 [HeartbeatError], exception: System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 ().

at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()

部署架构图

问题解答

多方查询无果后，咨询官方。得知这个是APIM 版本限制问题，升级APIM服务的版本（Azure自动维护）后，同一个公网IP地址最大的10个连接限制问题消失。

The issue is fixed starting version 0.23.1400.0 and above

参考资料

自托管网关概述：https://docs.azure.cn/zh-cn/api-management/self-hosted-gateway-overview

推荐链接

评论可见，请评论后查看内容，谢谢！！！评论后请刷新页面。

夸智网

【Azure API 管理】APIM Self-Host Gateway 自建本地环境中的网关数量超过10个且它们的出口IP为同一个时出现的429错误

AI绘画结婚证挂墙上，结婚证装裱

【Azure Developer - 密钥保管库】使用 Python Azure SDK 实现从 Azure Key Vault Certificate 中下载证书(PEM文件)

发表评论取消回复

夸智网

【Azure API 管理】APIM Self-Host Gateway 自建本地环境中的网关数量超过10个且它们的出口IP为同一个时出现的429错误

AI绘画结婚证挂墙上，结婚证装裱

【Azure Developer - 密钥保管库 】使用 Python Azure SDK 实现从 Azure Key Vault Certificate 中下载证书(PEM文件)

相关文章

发表评论取消回复

【Azure Developer - 密钥保管库】使用 Python Azure SDK 实现从 Azure Key Vault Certificate 中下载证书(PEM文件)