Free tools Windows power users keep installed
One-click scans. No signup required.
iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Use groupby(...).agg(...) when you want a compact summary with one row per group; use groupby(...).transform(...) when each original row must remain and receive a group-level value. Aggregation changes the shape of the data, while transformation returns results aligned to the rows you started with.
Choose the GroupBy operation that matches your output
A pandas GroupBy operation splits data into groups, applies a calculation, and combines the results. The method you choose determines whether the result is shorter than the source, remains row-aligned, or selects some of the original records.
| What you need | Method | Typical result |
|---|---|---|
| Summary values for each group | agg, aggregate, or a built-in reduction such as sum() |
One row per group |
| A group statistic attached to every source row | transform |
Same index as the grouped source object |
| Original rows selected using group conditions | filter, head, tail, or nth |
A selection of source rows |
| Custom logic or output shape not covered by a more specific method | apply |
Depends on what the function returns |
For a standard summary, prefer agg or a built-in reduction. Reserve apply for cases where a more specific GroupBy method cannot express the operation; pandas notes that apply can be slower. See the pandas GroupBy reference and the documentation for GroupBy apply.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →Summarize groups with agg
Aggregation reduces each group to one or more summary values. Unless you request otherwise, the group labels become the result index; grouping on multiple keys usually produces a MultiIndex. Use as_index=False to keep group labels as ordinary columns, or call reset_index() on the result.
#1 Best Overall
This example summarizes points by team and counts non-missing player values. It also retains the record with a missing team by setting dropna=False:
import pandas as pd
df = pd.DataFrame({
"team": ["A", "B", "A", None],
"player": ["Mina", "Noah", "Omar", "Pia"],
"points": [10, 7, 14, 8],
})
summary = (
df.groupby("team", as_index=False, dropna=False)
.agg(points_mean=("points", "mean"), player_count=("player", "count"))
)The result has one row per team group, including the NA-key group. The aggregation output is intentionally shorter than the input when several records share a team. Named aggregation lets you choose clear output column names, as in points_mean and player_count. See the DataFrameGroupBy.agg reference.
Keep every row with transform
Use transform when a group calculation must be repeated or otherwise aligned to every source row. For example, the team mean can be added as a new column without reducing the DataFrame:
df["team_mean"] = df.groupby("team", dropna=False)["points"].transform("mean")df["team_mean"] has one value for each row of df, including the row whose team is missing. The aggregation above produces group summaries; this transformation preserves row alignment. That distinction matters when assigning results: do not assign a shorter aggregated Series to the full DataFrame as if its values still correspond one-for-one to the original rows. If you need to attach a separate summary, join it using explicit group keys and check the resulting index alignment.
The transform reference describes same-indexed results. In pandas 2.0, DataFrame-returning transforms changed to align their result index with the input index; the documentation notes that calling .to_numpy() avoids that alignment when raw positional behavior is intended.
Keep group labels out of the index
For DataFrame aggregation, set as_index=False in groupby when the group keys should remain regular columns. Alternatively, use the default indexed result and call reset_index():
summary = df.groupby("team").agg(points_mean=("points", "mean")).reset_index()as_index controls aggregation output; it does not change transform‘s row alignment. If grouping by several columns, the default aggregation index contains the group keys as index levels, which reset_index() converts back into columns.
Understand missing keys and row order
Missing group keys
By default, groupby uses dropna=True, so rows with NA in a grouping key do not belong to a group and are omitted from grouped results. Set dropna=False when those rows should form an NA-key group. Choose deliberately: a grouped result with fewer records may reflect aggregation, excluded missing keys, or both.
Group order versus row order
By default, sort=True sorts group keys. Set sort=False to use the keys’ order of first appearance instead. This setting controls the order of groups, not the order of records inside each group: pandas preserves observation order within each group. See the pandas GroupBy sorting guide.
Categorical keys across pandas versions
When a grouping key is categorical, the observed option controls whether groups for unobserved category values appear. In pandas 3.0, observed defaults to True, so only observed categorical values are included. Older versions used a different default. Specify observed=True or observed=False explicitly when output must remain consistent across versions or include unobserved categories. The current defaults and version notes are listed in the DataFrame.groupby reference.
Use apply only for genuinely custom work
apply can combine a function’s DataFrame, Series, or scalar result across groups, making it flexible when the operation does not fit aggregation, transformation, or filtration. Its combined output shape depends on the function’s return value, so inspect the resulting index and columns rather than assuming they match the input. The group_keys option affects index construction for apply results; its default became True in pandas 2.0. The apply API reference documents these behaviors and cautions that specific methods such as agg and transform can be faster.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallOutdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchQuick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
