How To Export All Metric Data

Apr 28, 2025 by ADMIN 30 views

Introduction

VictoriaMetrics is a highly scalable and efficient time-series database that provides a robust solution for storing and querying large amounts of metric data. However, when trying to export all metric data for a specified time range through the /api/v1/export interface, you may encounter errors. In this article, we will explore the possible causes of these errors and provide guidance on how to configure VictoriaMetrics to successfully export all metric data.

Understanding the Error Message

The error message you provided indicates that the /api/v1/export interface returns an error when trying to export data for a specific time range. The error message is as follows:

error when exporting data on the time range (start=1742968800000, end=1742972400000): 
cannot fetch data for "accountID=0, projectID=0, filters=[{__name__!=\"\"}], 
timeRange=[2025-03-26T06:00:00Z..2025-03-26T07:00:00Z]": 
error occured during search: cannot fetch query results from vmstorage nodes: 
cannot perform search on vmstorage 192.168.159.133:8401: 
cannot execute funcName="search_v7" on vmstorage "192.168.159.133:8401": 
error when searching for tagFilters=[{AccountID=0,ProjectID=0,__name__=~".+"}] 
on the time range [2025-03-26T06:00:00Z..2025-03-26T07:00:00Z]: 
error when searching for metricIDs in the previous indexdb: 
the number of matching timeseries exceeds 2033328; 
either narrow down the search or increase -search.max* 
command-line flag values (the most likely limit is -search.maxUniqueTimeseries); 
see https://docs.victoriametrics.com/#resource-usage-limits

Troubleshooting Steps

To troubleshoot the issue, let's go through the error message step by step:

Check the Time Range: The error message indicates that the time range specified is causing the issue. Try to narrow down the time range to a smaller interval and see if the export is successful.
Verify the Filters: The error message mentions that the filters specified are causing the issue. Try to remove or modify the filters to see if the export is successful.
Check the vmstorage Configuration: The error message indicates that the vmstorage configuration is causing the issue. Check the vmstorage configuration to ensure that it is correctly set up and configured.
Increase the -search.maxUniqueTimeseries Value: The error message suggests increasing the -search.maxUniqueTimeseries value to resolve the issue. Try increasing this value and see if the export is successful.

Configuring VictoriaMetrics for Successful Export

To configure VictoriaMetrics for successful export, follow these steps:

Increase the -search.maxUniqueTimeseries Value: Increase the -search.maxUniqueTimeseries value to a higher number, such as 10000000. This will allow VictoriaMetrics to handle a larger number of unique timeseries.
Increase the -search.maxValuesPerSeries Value: Increase the -search.maxValuesPerSeries value to a higher number, such as 10000000. This will allow VictoriaMetrics to handle a larger number of values per series.
Increase the -search.maxValuesPerQuery Value: Increase the -search.maxValuesPerQuery value to a higher number, such as 10000000. This will allow VictoriaMetrics to handle a larger number of values per query.
Verify the vmstorage Configuration: Verify that the vmstorage configuration is correctly set up and configured.

Conclusion

In conclusion, the error message you provided indicates that the /api/v1/export interface is returning an error when trying to export data for a specific time range. To troubleshoot the issue, try narrowing down the time range, verifying the filters, checking the vmstorage configuration, and increasing the -search.maxUniqueTimeseries value. By following these steps, you should be able to configure VictoriaMetrics for successful export of all metric data.

Additional Resources

For more information on troubleshooting VictoriaMetrics, refer to the following resources:

Configuring VictoriaMetrics for Large-Scale Data Export

When dealing with large-scale data export, it's essential to configure VictoriaMetrics to handle the increased load. Here are some tips to help you configure VictoriaMetrics for large-scale data export:

Increase the -search.maxUniqueTimeseries Value: Increase the -search.maxUniqueTimeseries value to a higher number, such as 10000000. This will allow VictoriaMetrics to handle a larger number of unique timeseries.
Increase the -search.maxValuesPerSeries Value: Increase the -search.maxValuesPerSeries value to a higher number, such as 10000000. This will allow VictoriaMetrics to handle a larger number of values per series.
Increase the -search.maxValuesPerQuery Value: Increase the -search.maxValuesPerQuery value to a higher number, such as 10000000. This will allow VictoriaMetrics to handle a larger number of values per query.
Verify the vmstorage Configuration: Verify that the vmstorage configuration is correctly set up and configured.
Monitor Resource Usage: Monitor resource usage to ensure that VictoriaMetrics is not consuming excessive resources.

By following these tips, you should be able to configure VictoriaMetrics for large-scale data export.

Best Practices for Configuring VictoriaMetrics

Here are some best practices to keep in mind when configuring VictoriaMetrics:

Monitor Resource Usage: Monitor resource usage to ensure that VictoriaMetrics is not consuming excessive resources.
Verify the vmstorage Configuration: Verify that the vmstorage configuration is correctly set up and configured.
Increase the -search.maxUniqueTimeseries Value: Increase the -search.maxUniqueTimeseries value to a higher number, such as 10000000. This will allow VictoriaMetrics to handle a larger number of unique timeseries.
Increase the -search.maxValuesPerSeries Value: Increase the -search.maxValuesPerSeries value to a higher number, such as 10000000. This will allow VictoriaMetrics to handle a larger number of values per series.
Increase the -search.maxValuesPerQuery Value: Increase the -search.maxValuesPerQuery value to a higher number, such as 10000000. This will allow VictoriaMetrics to handle a larger number of values per query.

Q: What is the maximum number of unique timeseries that VictoriaMetrics can handle?

A: The maximum number of unique timeseries that VictoriaMetrics can handle is determined by the -search.maxUniqueTimeseries value. By default, this value is set to 2033328. However, you can increase this value to a higher number, such as 10000000, to handle a larger number of unique timeseries.

Q: How do I increase the -search.maxUniqueTimeseries value?

A: To increase the -search.maxUniqueTimeseries value, you can add the following flag to your VictoriaMetrics configuration file:

-search.maxUniqueTimeseries=10000000

Q: What is the maximum number of values per series that VictoriaMetrics can handle?

A: The maximum number of values per series that VictoriaMetrics can handle is determined by the -search.maxValuesPerSeries value. By default, this value is set to 2033328. However, you can increase this value to a higher number, such as 10000000, to handle a larger number of values per series.

Q: How do I increase the -search.maxValuesPerSeries value?

A: To increase the -search.maxValuesPerSeries value, you can add the following flag to your VictoriaMetrics configuration file:

-search.maxValuesPerSeries=10000000

Q: What is the maximum number of values per query that VictoriaMetrics can handle?

A: The maximum number of values per query that VictoriaMetrics can handle is determined by the -search.maxValuesPerQuery value. By default, this value is set to 2033328. However, you can increase this value to a higher number, such as 10000000, to handle a larger number of values per query.

Q: How do I increase the -search.maxValuesPerQuery value?

A: To increase the -search.maxValuesPerQuery value, you can add the following flag to your VictoriaMetrics configuration file:

-search.maxValuesPerQuery=10000000

Q: How do I troubleshoot issues with VictoriaMetrics?

A: To troubleshoot issues with VictoriaMetrics, you can refer to the following resources:

Q: How do I monitor resource usage in VictoriaMetrics?

A: To monitor resource usage in VictoriaMetrics, you can use the following commands:

vmctl status: Displays the current status of VictoriaMetrics, including resource usage.
vmctl metrics: Displays metrics about VictoriaMetrics, including resource usage.

Q: How do I verify the vmstorage configuration in VictoriaMetrics?

A: To verify the vmstorage configuration in VictoriaMetrics, you can use the following command:

vmctl config: Displays the current configuration of VictoriaMetrics, including the vmstorage configuration.

Q: How do I increase the -search.maxUniqueTimeseries value in a production environment?

A: To increase the -search.maxUniqueTimeseries value in a production environment, you should follow these steps:

Backup your data: Before making any changes to your VictoriaMetrics configuration, make sure to backup your data.
Test the change: Test the change in a non-production environment to ensure that it does not cause any issues.
Apply the change: Apply the change to your production environment.
Monitor resource usage: Monitor resource usage to ensure that the change does not cause any issues.

By following these steps, you can safely increase the -search.maxUniqueTimeseries value in a production environment.