Azure Data Explorer is a fast and highly scalable data exploration service for log and telemetry data. It helps handle the many data streams emitted by modern software to collect, store, and analyze data. Azure Data Explorer is ideal for analyzing large volumes of diverse data from any data source, including websites, applications, and IoT devices. This data is used for diagnostics, monitoring, reporting, machine learning, and other analytics capabilities. Azure Data Explorer makes it simple to ingest this data and enable you to do complex, ad hoc data queries in seconds.
Use the Azure Public cloud integration to discover and collect metrics against the Azure Kusto Cluster.
External reference
Azure Monitor - Platform metrics
Setup
To set up the Azure integration and discover the Azure service, go to Azure Integration Discovery Profile and select Kusto Clusters.
Event support
- Supported
- Configure Azure Events in OpsRamp Azure Integration Discovery Profile.
Supported metrics
| OpsRamp Metric | Azure Metric | Metric Display Name | Unit | Aggregation Type | Description |
|---|---|---|---|---|---|
| azure_kusto_cluster_CacheUtilizationFactor | CacheUtilizationFactor | Cache utilization factor | Percent | Average | Percentage of utilized disk space dedicated for hot cache in the cluster. 100% means that the disk space assigned to hot data is optimally utilized. No action is needed in terms of the cache size. More than 100% means that the cluster's disk space is not large enough to accommodate the hot data, as defined by your caching policies. To ensure that sufficient space is available for all the hot data, the amount of hot data needs to be reduced or the cluster needs to be scaled out. Enabling auto scale is recommended |
| azure_kusto_cluster_CPU | CPU | CPU | Percent | Average | CPU utilization level |
| azure_kusto_cluster_FollowerLatency | FollowerLatency | FollowerLatency | MilliSeconds | Average | The follower databases synchronize changes in the leader databases. Because of the synchronization, there's a data lag of a few seconds to a few minutes in data availability.This metric measures the length of the time lag. The time lag depends on the overall size of the leader database metadata.This is a cluster level metrics: the followers catch metadata of all databases that are followed. This metric represents the latency of the process |
| azure_kusto_cluster_IngestionUtilization | IngestionUtilization | Ingestion utilization | Percent | Average | Ratio of used ingestion slots in the cluster |
| azure_kusto_cluster_InstanceCount | InstanceCount | Instance Count | Count | Average | Total instance count |
| azure_kusto_cluster_KeepAlive | KeepAlive | Keep alive | Count | Average | Sanity check indicates the cluster responds to queries |
| azure_kusto_cluster_TotalNumberOfExtents | TotalNumberOfExtents | Total number of extents | Count | Average | Total number of data extents |
| azure_kusto_cluster_TotalNumberOfThrottledCommands | TotalNumberOfThrottledCommands | Total number of throttled commands | Count | Average | Total number of throttled commands |
| azure_kusto_cluster_ContinuousExportMaxLatenessMinutes | ContinuousExportMaxLatenessMinutes | Continuous Export Max Lateness | Count | Maximum | The lateness (in minutes) reported by the continuous export jobs in the cluster |
| azure_kusto_cluster_ContinuousExportNumOfRecordsExported | ContinuousExportNumOfRecordsExported | Continuous export - num of exported records | Count | Total | Number of records exported, fired for every storage artifact written during the export operation |
| azure_kusto_cluster_ContinuousExportPendingCount | ContinuousExportPendingCount | Continuous Export Pending Count | Count | Maximum | The number of pending continuous export jobs ready for execution |
| azure_kusto_cluster_ContinuousExportResult | ContinuousExportResult | Continuous Export Result | Count | Count | Indicates whether Continuous Export succeeded or failed |
| azure_kusto_cluster_ExportUtilization | ExportUtilization | Export Utilization | Percent | Maximum | Export utilization |
| azure_kusto_cluster_BatchBlobCount | BatchBlobCount | Batch Blob Count | Count | Average | Number of data sources in an aggregated batch for ingestion |
| azure_kusto_cluster_BatchDuration | BatchDuration | Batch Duration | Seconds | Average | The duration of the aggregation phase in the ingestion flow |
| azure_kusto_cluster_BatchesProcessed | BatchesProcessed | Batches Processed | Count | Total | Number of batches aggregated for ingestion. Batching Type: whether the batch reached batching time, data size or number of files limit set by batching policy |
| azure_kusto_cluster_BatchSize | BatchSize | Batch Size | Bytes | Average | Uncompressed expected data size in an aggregated batch for ingestion |
| azure_kusto_cluster_BlobsDropped | BlobsDropped | Blobs Dropped | Count | Total | Number of blobs permanently rejected by a component |
| azure_kusto_cluster_BlobsProcessed | BlobsProcessed | Blobs Processed | Count | Total | Number of blobs processed by a component |
| azure_kusto_cluster_BlobsReceived | BlobsReceived | Blobs Received | Count | Total | Number of blobs received from input stream by a component |
| azure_kusto_cluster_DiscoveryLatency | DiscoveryLatency | Discovery Latency | Seconds | Average | Reported by data connections (if exist). Time in seconds from when a message is enqueued or event is created until it is discovered by data connection. This time is not included in the Azure Data Explorer total ingestion duration |
| azure_kusto_cluster_EventsDropped | EventsDropped | Events Dropped | Count | Total | Number of events dropped permanently by data connection. An Ingestion result metric with a failure reason will be sent |
| azure_kusto_cluster_EventsProcessed | EventsProcessed | Events Processed | Count | Total | Number of events processed by the cluster |
| azure_kusto_cluster_EventsReceived | EventsReceived | Events Received | Count | Total | Number of events received by data connection |
| azure_kusto_cluster_IngestionLatencyInSeconds | IngestionLatencyInSeconds | Ingestion Latency | Seconds | Average | Latency of data ingested, from the time the data was received in the cluster until it's ready for query. The ingestion latency period depends on the ingestion scenario |
| azure_kusto_cluster_IngestionResult | IngestionResult | Ingestion result | Count | Total | Total number of sources that either failed or succeeded to be ingested. Splitting the metric by status, you can get detailed information about the status of the ingestion operations |
| azure_kusto_cluster_IngestionVolumeInMB | IngestionVolumeInMB | Ingestion Volume | Bytes | Total | Overall volume of ingested data to the cluster |
| azure_kusto_cluster_QueueLength | QueueLength | Queue Length | Count | Average | Number of pending messages in a component's queue |
| azure_kusto_cluster_QueueOldestMessage | QueueOldestMessage | Queue Oldest Message | Count | Average | Time in seconds from when the oldest message in queue was inserted |
| azure_kusto_cluster_ReceivedDataSizeBytes | ReceivedDataSizeBytes | Received Data Size Bytes | Bytes | Average | Size of data received by data connection. This is the size of the data stream, or of raw data size if provided |
| azure_kusto_cluster_StageLatency | StageLatency | Stage Latency | Seconds | Average | Cumulative time from when a message is discovered until it is received by the reporting component for processing (discovery time is set when message is enqueued for ingestion queue, or when discovered by data connection) |
| azure_kusto_cluster_MaterializedViewAgeMinutes | MaterializedViewAgeMinutes | Materialized View Age | Count | Average | The materialized view age in minutes |
| azure_kusto_cluster_MaterializedViewAgeSeconds | MaterializedViewAgeSeconds | Materialized View Age | Seconds | Average | The materialized view age in seconds |
| azure_kusto_cluster_MaterializedViewDataLoss | MaterializedViewDataLoss | Materialized View Data Loss | Count | Maximum | Indicates potential data loss in materialized view |
| azure_kusto_cluster_MaterializedViewExtentsRebuild | MaterializedViewExtentsRebuild | Materialized View Extents Rebuild | Count | Average | Number of extents rebuild |
| azure_kusto_cluster_MaterializedViewHealth | MaterializedViewHealth | Materialized View Health | Count | Average | The health of the materialized view (1 for healthy, 0 for non-healthy) |
| azure_kusto_cluster_MaterializedViewRecordsInDelta | MaterializedViewRecordsInDelta | Materialized View Records In Delta | Count | Average | The number of records in the non-materialized part of the view |
| azure_kusto_cluster_MaterializedViewResult | MaterializedViewResult | Materialized View Result | Count | Average | The result of the materialization process |
| azure_kusto_cluster_PartitioningPercentage | PartitioningPercentage | Partitioning Percentage | Percent | Average | Percentage of records partitioned versus total number of records |
| azure_kusto_cluster_PartitioningPercentageHot | PartitioningPercentageHot | Partitioning Percentage Hot | Percent | Average | Percentage of records partitioned versus total number of records (in hot / cached extents only) |
| azure_kusto_cluster_ProcessedPartitionedRecords | ProcessedPartitionedRecords | Processed Partitioned Records | Count | Average | Number of records partitioned in measured time window |
| azure_kusto_cluster_QueryDuration | QueryDuration | Query duration | MilliSeconds | Average | Queries duration in seconds |
| azure_kusto_cluster_QueryResult | QueryResult | Query Result | Count | Count | Total number of queries |
| azure_kusto_cluster_TotalNumberOfConcurrentQueries | TotalNumberOfConcurrentQueries | Total number of concurrent queries | Count | Average | Total number of concurrent queries |
| azure_kusto_cluster_TotalNumberOfThrottledQueries | TotalNumberOfThrottledQueries | Total number of throttled queries | Count | Average | Total number of throttled queries |
| azure_kusto_cluster_WeakConsistencyLatency | WeakConsistencyLatency | Weak consistency latency | Seconds | Average | The max latency between the previous metadata sync and the next one (in DB/node scope) |
| azure_kusto_cluster_StreamingIngestDataRate | StreamingIngestDataRate | Streaming Ingest Data Rate | Bytes | Average | Streaming ingest data rate |
| azure_kusto_cluster_StreamingIngestDuration | StreamingIngestDuration | Streaming Ingest Duration | MilliSeconds | Average | Streaming ingest duration in milliseconds |
| azure_kusto_cluster_StreamingIngestResults | StreamingIngestResults | Streaming Ingest Result | Count | Count | Streaming ingest result |
| azure_kusto_cluster_StreamingIngestUtilization | StreamingIngestUtilization | Streaming Ingest Utilization | Percent | Average | Streaming Ingest Utilization is the percentage of actual concurrent streaming ingestion requests performed, compared to the maximum number of concurrent streaming ingestion requests |