Metrics

Metrics Introduction

Metrics in Taurus' system are provided with Prometheus, an open-source monitoring system with a dimensional data model, flexible query language, efficient time series database and modern alerting approach.

Taurus mainly uses Prometheus to collect and store metrics as time series data. This means that metrics information is stored with the timestamp at which it was recorded, alongside optional key-value pairs called labels.

In this page, we will explain the different metric families, with thorough explanations and detailed representations.

Metrics Family and Description

Go Metrics


Metric FamilyMetric Description
go_build_infoBuild information about the main Go module.
go_gc_duration_secondsA summary of the wall-time pause (stop-the-world) duration in garbage collection cycles.
go_gc_gogc_percentHeap size target percentage configured by the user, otherwise 100. This value is set by the GOGC environment variable, and the runtime/debug.SetGCPercent function. Sourced from /gc/gogc:percent
go_gc_gomemlimit_bytesGo runtime memory limit configured by the user, otherwise math.MaxInt64. This value is set by the GOMEMLIMIT environment variable, and the runtime/debug.SetMemoryLimit function. Sourced from /gc/gomemlimit:bytes
go_goroutinesNumber of goroutines that currently exist.
go_infoInformation about the Go environment.
go_memstats_alloc_bytesNumber of bytes allocated in heap and currently in use. Equals to /memory/classes/heap/objects:bytes.
go_memstats_alloc_bytes_totalTotal number of bytes allocated in heap until now, even if released already. Equals to /gc/heap/allocs:bytes.
go_memstats_buck_hash_sys_bytesNumber of bytes used by the profiling bucket hash table. Equals to /memory/classes/profiling/buckets:bytes.
go_memstats_frees_totalTotal number of heap objects frees. Equals to /gc/heap/frees:objects + /gc/heap/tiny/allocs:objects.
go_memstats_gc_sys_bytesNumber of bytes used for garbage collection system metadata. Equals to /memory/classes/metadata/other:bytes.
go_memstats_heap_alloc_bytesNumber of heap bytes allocated and currently in use, same as go_memstats_alloc_bytes. Equals to /memory/classes/heap/objects:bytes.
go_memstats_heap_idle_bytesNumber of heap bytes waiting to be used. Equals to /memory/classes/heap/released:bytes + /memory/classes/heap/free:bytes.
go_memstats_heap_inuse_bytesNumber of heap bytes that are in use. Equals to /memory/classes/heap/objects:bytes + /memory/classes/heap/unused:bytes.
go_memstats_heap_objectsNumber of currently allocated objects. Equals to /gc/heap/objects:objects.
go_memstats_heap_released_bytesNumber of heap bytes released to OS. Equals to /memory/classes/heap/released:bytes.
go_memstats_heap_sys_bytesNumber of heap bytes obtained from system. Equals to /memory/classes/heap/objects:bytes + /memory/classes/heap/unused:bytes + /memory/classes/heap/released:bytes + /memory/classes/heap/free:bytes.
go_memstats_last_gc_time_secondsNumber of seconds since 1970 of last garbage collection.
go_memstats_mallocs_totalTotal number of heap objects allocated, both live and gc-ed. Semantically a counter version for go_memstats_heap_objects gauge. Equals to /gc/heap/allocs:objects + /gc/heap/tiny/allocs:objects.
go_memstats_mcache_inuse_bytesNumber of bytes in use by mcache structures. Equals to /memory/classes/metadata/mcache/inuse:bytes.
go_memstats_mcache_sys_bytesNumber of bytes used for mcache structures obtained from system. Equals to /memory/classes/metadata/mcache/inuse:bytes + /memory/classes/metadata/mcache/free:bytes.
go_memstats_mspan_inuse_bytesNumber of bytes in use by mspan structures. Equals to /memory/classes/metadata/mspan/inuse:bytes.
go_memstats_mspan_sys_bytesNumber of bytes used for mspan structures obtained from system. Equals to /memory/classes/metadata/mspan/inuse:bytes + /memory/classes/metadata/mspan/free:bytes.
go_memstats_next_gc_bytesNumber of heap bytes when next garbage collection will take place. Equals to /gc/heap/goal:bytes.
go_memstats_other_sys_bytesNumber of bytes used for other system allocations. Equals to /memory/classes/other:bytes.
go_memstats_stack_inuse_bytesNumber of bytes obtained from system for stack allocator in non-CGO environments. Equals to /memory/classes/heap/stacks:bytes.
go_memstats_stack_sys_bytesNumber of bytes obtained from system for stack allocator. Equals to /memory/classes/heap/stacks:bytes + /memory/classes/os-stacks:bytes.
go_memstats_sys_bytesNumber of bytes obtained from system. Equals to /memory/classes/total:byte.
go_sched_gomaxprocs_threadsThe current runtime.GOMAXPROCS setting, or the number of operating system threads that can execute user-level Go code simultaneously. Sourced from /sched/gomaxprocs:threads.
go_sql_idle_connectionsThe number of idle connections.
go_sql_in_use_connectionsThe number of connections currently in use.
go_sql_max_idle_closed_totalThe total number of connections closed due to SetMaxIdleConns.
go_sql_max_idle_time_closed_totalThe total number of connections closed due to SetConnMaxIdleTime.
go_sql_max_lifetime_closed_totalThe total number of connections closed due to SetConnMaxLifetime.
go_sql_max_open_connectionsMaximum number of open connections to the database.
go_sql_open_connectionsThe number of established connections both in use and idle.
go_sql_wait_count_totalThe total number of connections waited for.
go_sql_wait_duration_seconds_totalThe total time blocked waiting for a new connection.
go_threadsNumber of OS threads created.

GRPC Metrics

Metric FamilyMetric Description
grpc_server_handled_totalTotal number of RPCs completed on the server, regardless of success or failure.
grpc_server_handling_secondsSummary of response latency (seconds) of gRPC that had been application-level handled by the server.

Taurus validatord metrics

Metric FamilyMetric Description
tg_validatord_cryptocompareClient_APICall_totalTotal number of API calls by the cryptocompare client.
tg_validatord_database_request_timing_secondsTime taken by database requests.
tg_validatord_infoInfo about the particular instance.
tg_validatord_job_wallet_elapsed_time_nsElapsed time for job wallet operations in nanoseconds.
tg_validatord_log_event_counterCounts the number of log events.
tg_validatord_pq_error_counterTotal number of database errors from pq.

Taurus Job Metrics

Metric FamilyMetric Description
tg_job_job_execution_time_in_secondsSummary of job execution time. An observation is created for each job execution.
tg_job_job_stateState of job: 0=unknown, 1=disabled, 2=enabled.
tg_job_job_status_countersCount the status occurrences.
tgJob_controller_stateState of the controller: 0=unknown, 1=stopped, 2=started.
tgJob_infoInfo about the job library.

Metric Family and representation

Metric NameMetric Representation
go_build_infogauge
go_gc_duration_secondssummary
go_gc_gogc_percentgauge
go_gc_gomemlimit_bytesgauge
go_goroutinesgauge
go_infogauge
go_memstats_alloc_bytesgauge
go_memstats_alloc_bytes_totalcounter
go_memstats_buck_hash_sys_bytesgauge
go_memstats_frees_totalcounter
go_memstats_gc_sys_bytesgauge
go_memstats_heap_alloc_bytesgauge
go_memstats_heap_idle_bytesgauge
go_memstats_heap_inuse_bytesgauge
go_memstats_heap_objectsgauge
go_memstats_heap_released_bytesgauge
go_memstats_heap_sys_bytesgauge
go_memstats_last_gc_time_secondsgauge
go_memstats_mallocs_totalcounter
go_memstats_mcache_inuse_bytesgauge
go_memstats_mcache_sys_bytesgauge
go_memstats_mspan_inuse_bytesgauge
go_memstats_mspan_sys_bytesgauge
go_memstats_next_gc_bytesgauge
go_memstats_other_sys_bytesgauge
go_memstats_stack_inuse_bytesgauge
go_memstats_stack_sys_bytesgauge
go_memstats_sys_bytesgauge
go_sched_gomaxprocs_threadsgauge
go_sql_idle_connectionsgauge
go_sql_in_use_connectionsgauge
go_sql_max_idle_closed_totalcounter
go_sql_max_idle_time_closed_totalcounter
go_sql_max_lifetime_closed_totalcounter
go_sql_max_open_connectionsgauge
go_sql_open_connectionsgauge
go_sql_wait_count_totalcounter
go_sql_wait_duration_seconds_totalcounter
go_threadsgauge
grpc_server_handled_totalcounter
grpc_server_handling_secondssummary
tg_job_job_execution_time_in_secondssummary
tg_job_job_stategauge
tg_job_job_status_counterscounter
tg_validatord_cryptocompareClient_APICall_totalcounter
tg_validatord_database_request_timing_secondssummary
tg_validatord_infogauge
tg_validatord_job_wallet_elapsed_time_nsgauge
tg_validatord_log_event_countercounter
tg_validatord_pq_error_countercounter
tgJob_controller_stategauge
tgJob_infogauge


  © 2018-2023 Taurus SA. All rights reserved.