iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
How do I downsample data in Python without losing important information? First decide what “downsample” means for your task: summarize timestamped records into coarser time bins, lower the sample rate of a regular digital signal, or reduce plotted points for display. Those operations preserve different things. For time-indexed business or sensor data, use a meaningful aggregation; for a regularly sampled signal, filter before removing samples; for a large chart, use display-oriented thinning and keep the full data for analysis.
Choose a method based on what you need to preserve
| Goal and input | Python starting point | What to check |
|---|---|---|
| Summarize timestamped records into fixed time bins | pandas.Series.resample or DataFrame.resample, followed by an aggregation |
Choose a meaningful statistic, time-bin frequency, boundary convention, time zone, and missing-value handling. This is time-based grouping, not signal filtering. Pandas time-series documentation |
| Reduce a regularly sampled signal by an integer factor | scipy.signal.decimate(x, q) |
It applies an anti-aliasing filter before reduction. Check filter and phase needs; SciPy recommends multiple calls for IIR factors above 13. SciPy decimate reference |
| Resample an evenly sampled, periodic signal to a chosen number of points | scipy.signal.resample(x, num) |
FFT-based resampling supports arbitrary output lengths but assumes periodic continuation, which can cause edge effects for non-periodic records. SciPy resample reference |
| Change an evenly sampled signal’s rate by a rational factor | scipy.signal.resample_poly(x, up, down) |
Uses an FIR polyphase method; filter and boundary settings matter. The cited documentation is for a development version, so check your installed SciPy version. SciPy resample_poly reference |
| Display a very large time series in an interactive chart | Plotly-Resampler or a visualization-oriented package such as tsdownsample | Choose aggregation for visible shape and features. A plotted subset is not automatically suitable for statistical analysis. Plotly-Resampler paper; tsdownsample paper |
There is no single best downsampling algorithm. A mean can summarize typical values while hiding brief peaks; a sum can preserve totals but not waveform shape; a low-pass filter controls aliasing but does not produce the same result as time-bin aggregation. Decide whether the output is for analysis, signal processing, or display before choosing a method.
Aggregate timestamped records with pandas
Pandas resample groups observations into time intervals. Start with a datetime-like index, specify a frequency, then select an aggregation that matches the meaning of the data. For example, if df has a DatetimeIndex and a numeric column named value, this calculates hourly means:
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →hourly = df["value"].resample("1h").mean()Choose the aggregation deliberately
- For a typical level over each interval, use a mean when that summary is meaningful.
- For quantities that accumulate, such as interval event totals, a sum or count may be more appropriate.
- For peak monitoring, inspect minimum and maximum values rather than assuming the mean captures important events.
The right statistic depends on what each original observation represents. Averages of instantaneous readings, for example, are not interchangeable with totals of events or usage.
#1 Best Overall
Define interval edges and missing values
Set closed and label intentionally when readings fall on bin boundaries, especially for reporting or billing periods. These options control which interval includes an edge observation and which timestamp labels the resulting bin. Consider the index time zone and the intended calendar convention as well.
Check empty bins and missing observations. A generated NaN is not a measured zero, and filling it with zero changes the meaning of the result. Resampling can also create intermediate rows when the input is sparse; avoid generating a denser index than the task requires. Pandas documents resampling as time-based grouping and describes the interval options in its time-series guide.
Reduce a regular signal with an anti-aliasing filter
For a digital signal sampled at regular intervals, simply selecting every fourth value with x[::4] discards samples without first suppressing frequencies that can fold into lower frequencies. SciPy’s signal.decimate applies an anti-aliasing filter before reducing the signal by an integer factor q:
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsfrom scipy import signal
y_small = signal.decimate(x, q=4, zero_phase=True)The example reduces an equidistant signal by a factor of four. SciPy’s reference describes the operation as “Downsample the signal after applying an anti-aliasing filter.” The default filter is an order-8 Chebyshev type I IIR filter; with ftype="fir", the documented alternative is a 30-point Hamming-window FIR filter. By default, zero_phase=True avoids phase shift, which is generally useful when phase displacement is unwanted. For IIR factors greater than 13, SciPy recommends applying decimation in multiple calls. See the SciPy decimate reference for API details.
Choose Fourier or polyphase resampling when the rate ratio is not just an integer
Both methods below are for evenly sampled signals. They address rate conversion, not aggregation of irregular timestamped records. The choice depends in part on whether the signal is periodic, what output rate is needed, and how the finite record’s boundaries should be treated.
Fourier resampling for periodic signals and arbitrary output lengths
from scipy.signal import resample
y_new = resample(x, num=target_count)signal.resample changes the FFT length by shortening or zero-padding it, allowing a chosen output count. Its key assumption is that the signal continues periodically beyond the observed record. If the last part of a non-periodic recording does not join smoothly to the first, the implied wraparound can create edge behavior that is not representative of the real signal. FFT lengths that are prime or have few prime factors may also take longer to process. See the SciPy resample reference.
Polyphase resampling for rational rate changes
from scipy.signal import resample_poly
y_new = resample_poly(x, up=1, down=4)This example reduces the rate by four. In general, resample_poly changes sample spacing by down / up, using a low-pass FIR filter in a polyphase implementation. It can be faster than Fourier resampling for some large or prime-sized inputs and factor combinations, but that is not a universal speed guarantee. Review the filter and padding behavior for the signal’s boundaries. If you provide custom filter coefficients, design them for the upsampled rate; symmetric odd-length coefficients can support zero-phase centering.
Recommended Free Tools
The available reference is SciPy 2.0.0 development documentation, not a stable-version guarantee. Confirm the API and behavior against the version installed in your environment. See the SciPy resample_poly reference.
Best Value
Reduce chart points without replacing the analysis data
Drawing every point in a very large time series can make an interactive chart difficult to use. Viewport-aware approaches return or aggregate points for the visible range, so the displayed subset can change as a reader zooms or pans. The Plotly-Resampler paper describes aggregation that responds to the current graph view; the tsdownsample paper presents a CPU-based, in-memory Python package with Rust SIMD and multithreading and evaluates selected algorithms and integrations. These papers report designs and experiments, not guaranteed performance on every machine or universal preservation of every signal feature.
For a visualization, decide whether the display must retain extrema, transitions, overall shape, or another feature. A method that keeps visible peaks may not preserve the underlying distribution, while averaging can conceal short-lived extremes. Compare the chart against the raw data at spikes, gaps, and transitions. Treat the reduced series as a rendering aid unless it has separately been validated for the analysis question.
Quick Recap
Validate the reduced output against the original
- Confirm the input geometry: irregular timestamps call for a different approach from evenly spaced signal samples.
- Check whether the output preserves the property your task needs—totals, averages, bandwidth, extrema, trends, or visible chart shape.
- Inspect timestamps, bin alignment, time zones, gaps, empty intervals, and the start and end of the record.
- For signal-rate reduction, verify that filtering and phase behavior meet the use case; for visualization, inspect peaks and transitions at the scales readers will view.
- Keep the raw observations when later analysis may need information removed by aggregation, filtering, or point reduction. Record the method and its parameters so the reduced result can be interpreted.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →
