Promitor Scraper Agent has encountered an unexpected error. #2138

nikitakogan1 · 2022-09-29T13:24:42Z

nikitakogan1
Sep 29, 2022

Hi ,

I am trying to install promitor on my Azure K8S cluster.

I use a sample configuration, with only 1 metric:

values file:

version: v1
azureMetadata:
  tenantId: tenant
  subscriptionId: subscription
  resourceGroupName: resource-geoup
  cloud: Global
metricDefaults:
  aggregation:
    # azure metrics need 5 minutes aggregation interval (some won't work @1 minute)
    interval: 00:05:00
  scraping:
  # Every minute
    schedule: "0 * * ? * *"
metrics:
- name: azure_virtual_network_ddos_attack
description: "Indication whether or not there is a DDOS attack on the Azure Virtual Network"
resourceType: VirtualNetwork
azureMetricConfiguration:
  metricName: IfUnderDDoSAttack
  aggregation:
    type: Maximum
  resources: # Optional, required when no resource discovery is configured
  - virtualNetworkName: vnet-name

command for installation:

helm install promitor-agent-scraper promitor/promitor-agent-scraper \
               --set azureAuthentication.identity.mode='ServicePrincipal' \
               --set azureAuthentication.identity.id='xxxx' \
               --set azureAuthentication.identity.key='xxxxx' \
               --values values.yaml
               --namespace monitoring

output:


██████╗ ██████╗  ██████╗ ███╗   ███╗██╗████████╗ ██████╗ ██████╗
██╔══██╗██╔══██╗██╔═══██╗████╗ ████║██║╚══██╔══╝██╔═══██╗██╔══██╗
██████╔╝██████╔╝██║   ██║██╔████╔██║██║   ██║   ██║   ██║██████╔╝
██╔═══╝ ██╔══██╗██║   ██║██║╚██╔╝██║██║   ██║   ██║   ██║██╔══██╗
██║     ██║  ██║╚██████╔╝██║ ╚═╝ ██║██║   ██║   ╚██████╔╝██║  ██║
╚═╝     ╚═╝  ╚═╝ ╚═════╝ ╚═╝     ╚═╝╚═╝   ╚═╝    ╚═════╝ ╚═╝  ╚═╝
[12:51:31 INF] Booting up Promitor v2.7.0 running .NET .NET 6.0.6 - Thank you for using Promitor!
[12:51:31 INF] Running .NET 6.0.6 on Linux (alpine.3.14-x64 | Linux 5.10.114-garden-cloud-amd64 #1 SMP Debian 5.10.114-0gardenlinux1 (2022-05-09)).
[12:51:31 INF] Using configuration folder '/config/'
[12:51:32 FTL] Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it.
System.ArgumentNullException: Value cannot be null. (Parameter 'key')
   at System.Collections.Generic.Dictionary`2.FindValue(TKey key)
   at Microsoft.Extensions.DependencyInjection.SchedulingExtensions.GroupMetricsByScrapingInterval(MetricsDeclaration allMetrics) in /src/Promitor.Agents.Scraper/SchedulingExtensions.cs:line 59
   at Microsoft.Extensions.DependencyInjection.SchedulingExtensions.ScheduleMetricScraping(IServiceCollection services) in /src/Promitor.Agents.Scraper/SchedulingExtensions.cs:line 46
   at Promitor.Agents.Scraper.Startup.ConfigureServices(IServiceCollection services) in /src/Promitor.Agents.Scraper/Startup.cs:line 55
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Span`1& arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.AspNetCore.Hosting.ConfigureServicesBuilder.InvokeCore(Object instance, IServiceCollection services)
   at Microsoft.AspNetCore.Hosting.ConfigureServicesBuilder.<>c__DisplayClass9_0.<Invoke>g__Startup|0(IServiceCollection serviceCollection)
   at Microsoft.AspNetCore.Hosting.ConfigureServicesBuilder.Invoke(Object instance, IServiceCollection services)
   at Microsoft.AspNetCore.Hosting.ConfigureServicesBuilder.<>c__DisplayClass8_0.<Build>b__0(IServiceCollection services)
   at Microsoft.AspNetCore.Hosting.GenericWebHostBuilder.UseStartup(Type startupType, HostBuilderContext context, IServiceCollection services, Object instance)
   at Microsoft.AspNetCore.Hosting.GenericWebHostBuilder.<>c__DisplayClass13_0.<UseStartup>b__0(HostBuilderContext context, IServiceCollection services)
   at Microsoft.Extensions.Hosting.HostBuilder.CreateServiceProvider()
   at Microsoft.Extensions.Hosting.HostBuilder.Build()
   at Promitor.Agents.Scraper.Program.Main(String[] args) in /src/Promitor.Agents.Scraper/Program.cs:line 35

Can someone hint about what is wrong with my configuration?
Thanks!

Answered by tomkerkhove

Sep 30, 2022

This is typically related to the configuration not being valid and is being tracked to improve.

I think the problem is your indention:

metrics:
- name: azure_virtual_network_ddos_attack
description: "Indication whether or not there is a DDOS attack on the Azure Virtual Network"
resourceType: VirtualNetwork
azureMetricConfiguration:
  metricName: IfUnderDDoSAttack
  aggregation:
    type: Maximum
  resources: # Optional, required when no resource discovery is configured
  - virtualNetworkName: vnet-name

Instead, it should be something like this:

metrics:
- name: azure_virtual_network_ddos_attack
  description: "Indication whether or not there is a DDOS attack on the Azure Virtual Network"…

View full answer

tomkerkhove · 2022-09-30T07:24:50Z

tomkerkhove
Sep 30, 2022
Maintainer

This is typically related to the configuration not being valid and is being tracked to improve.

I think the problem is your indention:

metrics:
- name: azure_virtual_network_ddos_attack
description: "Indication whether or not there is a DDOS attack on the Azure Virtual Network"
resourceType: VirtualNetwork
azureMetricConfiguration:
  metricName: IfUnderDDoSAttack
  aggregation:
    type: Maximum
  resources: # Optional, required when no resource discovery is configured
  - virtualNetworkName: vnet-name

Instead, it should be something like this:

metrics:
- name: azure_virtual_network_ddos_attack
  description: "Indication whether or not there is a DDOS attack on the Azure Virtual Network"
  resourceType: VirtualNetwork
  azureMetricConfiguration:
    metricName: IfUnderDDoSAttack
    aggregation:
      type: Maximum
  resources: # Optional, required when no resource discovery is configured
  - virtualNetworkName: vnet-name

0 replies

nikitakogan1 · 2022-10-02T16:16:38Z

nikitakogan1
Oct 2, 2022
Author

Hi @tomkerkhove
Thanks for the quick answer. The issue seems to be resolved. But now i am facing a new issue.
Shouldn't service discovery be optional?


System.Exception: Resource Discovery is not configured
   at GuardNet.Guard.For[TException](Func`1 predicate, TException exception)
   at GuardNet.Guard.For[TException](Func`1 predicate, String message)
   at Promitor.Agents.Scraper.Discovery.ResourceDiscoveryClient..ctor(HttpClient httpClient, IOptionsMonitor`1 configuration, ILogger`1 logger) in /src/Promitor.Agents.Scraper/ResourceDiscoveryClient.cs:line 29
   at lambda_method19(Closure , IServiceProvider , Object[] )
   at Microsoft.Extensions.Http.DefaultTypedHttpClientFactory`1.CreateClient(HttpClient httpClient)
   at Microsoft.Extensions.DependencyInjection.HttpClientBuilderExtensions.AddTransientHelper[TClient](IServiceProvider s, IHttpClientBuilder builder)
   at Microsoft.Extensions.DependencyInjection.HttpClientBuilderExtensions.<>c__DisplayClass10_0`1.<AddTypedClientCore>b__0(IServiceProvider s)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass2_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.SchedulingExtensions.<>c__DisplayClass2_0.<ScheduleResourcesScraping>b__1(IServiceProvider jobServices) in /src/Promitor.Agents.Scraper/SchedulingExtensions.cs:line 96
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitIEnumerable(IEnumerableCallSite enumerableCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(Type serviceType)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.SchedulerServiceCollectionExtensions.<>c__DisplayClass6_0.<CreateInstance>b__0(IServiceProvider sp) in C:\projects\cronscheduler-aspnetcore\src\CronScheduler.Extensions\DependencyInjection\SchedulerServiceCollectionExtensions.cs:line 119
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitIEnumerable(IEnumerableCallSite enumerableCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(Type serviceType)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider)
   at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)
   at Promitor.Agents.Scraper.Program.Main(String[] args) in /src/Promitor.Agents.Scraper/Program.cs:line 52

3 replies

tomkerkhove Oct 3, 2022
Maintainer

This was fixed in #2104

I'm actually in process of releasing Scraper v2.8

nikitakogan1 Oct 3, 2022
Author

I managed to make it work with scraper chart version of 2.7.
Thanks for all the support and for this application.

tomkerkhove Oct 3, 2022
Maintainer

Happy to help!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Promitor Scraper Agent has encountered an unexpected error. #2138

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 2 comments 3 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Promitor Scraper Agent has encountered an unexpected error. #2138

nikitakogan1 Sep 29, 2022

Replies: 2 comments · 3 replies

tomkerkhove Sep 30, 2022 Maintainer

nikitakogan1 Oct 2, 2022 Author

tomkerkhove Oct 3, 2022 Maintainer

nikitakogan1 Oct 3, 2022 Author

tomkerkhove Oct 3, 2022 Maintainer

nikitakogan1
Sep 29, 2022

Replies: 2 comments 3 replies

tomkerkhove
Sep 30, 2022
Maintainer

nikitakogan1
Oct 2, 2022
Author

tomkerkhove Oct 3, 2022
Maintainer

nikitakogan1 Oct 3, 2022
Author

tomkerkhove Oct 3, 2022
Maintainer