SQL聚合函数核心用法与性能优化实战-代码聚汇网

SQL聚合函数核心用法与性能优化实战

SQL聚合函数GROUP BYHAVING

|难度：普通|阅读约 40 分钟

1. SQL聚合函数基础与应用场景

SQL聚合函数是数据分析师和开发人员日常工作中最常用的工具之一。它们允许我们对数据集进行汇总统计，从海量数据中提取有价值的信息。在实际业务场景中，聚合函数通常与GROUP BY子句配合使用，能够解决80%以上的基础数据分析需求。

1.1 核心聚合函数解析

COUNT() 函数可能是最常用的聚合函数，但很多人并不完全了解它的各种用法细节：

COUNT(*)：统计所有行数，包括NULL值
COUNT(column_name)：统计指定列非NULL值的数量
COUNT(DISTINCT column_name)：统计列中不同值的数量

实际工作中，统计用户数时一定要用COUNT(DISTINCT user_id)而非COUNT(user_id)，因为同一个用户可能在表中有多条记录。

SUM() 和 AVG() 函数在处理数值数据时尤为关键。我曾在一个电商项目中踩过坑：计算平均订单金额时直接使用AVG()，忽略了部分订单可能存在退款情况，导致数据失真。正确的做法应该是：

sql复制SELECT 
    SUM(amount - refund_amount) / COUNT(*) AS real_avg_order_amount
FROM orders

MAX() 和 MIN() 虽然简单，但在分析数据分布时非常有用。比如分析用户活跃时间段：

sql复制SELECT 
    MIN(login_time) AS first_login,
    MAX(login_time) AS last_login
FROM user_sessions
WHERE user_id = 12345

1.2 GROUP BY的深度理解

GROUP BY的常见误区是认为它只是简单的"分组"，实际上它是SQL执行顺序中的关键环节。一个完整的SQL查询执行顺序是：

FROM和JOIN确定数据源
WHERE筛选行
GROUP BY分组
HAVING筛选组
SELECT选择列
ORDER BY排序
LIMIT限制结果

我曾见过一个典型错误案例：开发人员试图在WHERE子句中使用聚合函数，导致语法错误。正确的做法是使用HAVING：

sql复制-- 错误写法
SELECT department, AVG(salary) 
FROM employees
WHERE AVG(salary) > 5000
GROUP BY department

-- 正确写法
SELECT department, AVG(salary) 
FROM employees
GROUP BY department
HAVING AVG(salary) > 5000

需要模型API调用？免费领10W Token，多模型网关一键接入 Claude、DeepSeek 等主流模型。

2. 实战题目深度解析

2.1 题目1：电影筛选与排序

原始解决方案已经很好地解决了问题，但我们可以进一步优化：

sql复制SELECT 
    id,
    movie,
    description,
    rating
FROM cinema
WHERE description <> 'boring'  -- 使用<>而非!=更符合SQL标准
  AND MOD(id, 2) = 1          -- 使用MOD函数更清晰
ORDER BY rating DESC
LIMIT 10;                     -- 添加分页限制

关键细节：

<>和!=在大多数数据库中功能相同，但<>是SQL标准写法
MOD(id, 2)比id % 2可读性更好，特别是在复杂表达式中
生产环境中一定要加LIMIT，避免返回过多数据影响性能

2.2 题目2：产品平均价格计算

这个题目展示了实际业务中常见的价格计算场景。原始解决方案很好，但我们可以增加更多业务考量：

sql复制SELECT 
    p.product_id,
    p.product_name,  -- 实际业务中通常会显示产品名称
    ROUND(
        COALESCE(
            SUM(p.price * u.units) / NULLIF(SUM(u.units), 0), 
            0
        ), 
        2
    ) AS average_price
FROM Prices p
LEFT JOIN UnitsSold u ON p.product_id = u.product_id 
    AND u.purchase_date BETWEEN p.start_date AND p.end_date
    AND u.is_valid = 1  -- 假设有标记有效销售的字段
GROUP BY p.product_id, p.product_name
HAVING SUM(u.units) > 0 OR SUM(u.units) IS NULL  -- 可选：过滤掉零销售产品

改进点解析：

使用COALESCE替代IFNULL，它是更通用的标准SQL函数
添加NULLIF防止除以零错误
增加了业务逻辑字段is_valid的检查
使用HAVING进行结果过滤

3. 连接查询的高级技巧

3.1 显式连接与隐式连接对比

原始内容已经很好地对比了两种连接方式，我想补充一些实际项目经验：

性能考虑：

现代数据库优化器对显式JOIN的处理通常更好
复杂查询中，显式JOIN能让优化器更准确地选择连接顺序
隐式连接在多表关联时可能导致笛卡尔积风险

团队规范：

新项目强制使用显式JOIN
旧系统维护时保持原有风格一致性
在存储过程中统一注释连接逻辑

特殊场景：

交叉连接(CROSS JOIN)使用隐式语法更简洁
自然连接(NATURAL JOIN)只能使用显式语法

3.2 题目3的扩展：连接性能优化

sql复制-- 优化后的显式连接查询
SELECT
    p.project_id,
    p.project_name,
    ROUND(AVG(e.experience_years), 2) AS average_years,
    COUNT(e.employee_id) AS team_size
FROM Project p
JOIN Employee e ON p.employee_id = e.employee_id
WHERE p.status = 'active'  -- 添加业务过滤条件
GROUP BY p.project_id, p.project_name
HAVING COUNT(e.employee_id) > 3  -- 只统计团队规模大于3的项目
ORDER BY average_years DESC;

优化点说明：

添加了项目状态过滤，符合实际业务需求
增加了团队规模统计
使用HAVING过滤小团队
按平均经验排序，更有业务意义

4. 复杂聚合与子查询

4.1 题目4：注册率计算的多种实现

原始解决方案很好，以下是几种替代方案及其适用场景：

方案1：使用CTE提高可读性

sql复制WITH total_users AS (
    SELECT COUNT(user_id) AS cnt FROM Users
)
SELECT
    r.contest_id,
    ROUND(COUNT(r.user_id) * 100.0 / (SELECT cnt FROM total_users), 2) AS percentage
FROM Register r
GROUP BY r.contest_id
ORDER BY percentage DESC, r.contest_id;

方案2：使用JOIN避免子查询

sql复制SELECT
    r.contest_id,
    ROUND(COUNT(r.user_id) * 100.0 / MAX(u.total_count), 2) AS percentage
FROM Register r
CROSS JOIN (SELECT COUNT(*) AS total_count FROM Users) u
GROUP BY r.contest_id
ORDER BY percentage DESC, r.contest_id;

方案3：使用窗口函数

sql复制SELECT DISTINCT
    contest_id,
    ROUND(
        COUNT(user_id) OVER (PARTITION BY contest_id) * 100.0 / 
        COUNT(user_id) OVER (),
        2
    ) AS percentage
FROM Register
ORDER BY percentage DESC, contest_id;

性能对比：

小数据量：三种方案差异不大
大数据量：方案2通常性能最好
可读性：方案1最易理解和维护

4.2 题目5：质量指标计算的陷阱

原始解决方案已经很好，我想强调几个容易出错的点：

常见错误1：忽略NULL值

sql复制-- 错误：如果rating或position有NULL，整个表达式结果为NULL
AVG(rating / position)

-- 正确：处理NULL值
AVG(CASE WHEN rating IS NOT NULL AND position IS NOT NULL 
         THEN rating / position 
         ELSE NULL END)

常见错误2：整数除法

sql复制-- 错误：在部分数据库中，整数除法会截断小数
SUM(rating < 3) / COUNT(*)

-- 正确：确保使用浮点数除法
SUM(rating < 3) * 1.0 / COUNT(*)

生产环境增强版：

sql复制SELECT
    query_name,
    ROUND(
        AVG(CASE WHEN position > 0 THEN rating * 1.0 / position ELSE NULL END),
        2
    ) AS quality,
    ROUND(
        100.0 * SUM(CASE WHEN rating < 3 THEN 1 ELSE 0 END) / COUNT(*),
        2
    ) AS poor_query_percentage,
    COUNT(*) AS query_count  -- 添加样本量参考
FROM Queries
WHERE query_name IS NOT NULL  -- 过滤脏数据
GROUP BY query_name
HAVING COUNT(*) >= 10  -- 只统计有足够样本的查询
ORDER BY quality DESC;

5. 聚合查询性能优化

5.1 索引策略

聚合查询性能很大程度上取决于正确的索引设计：

GROUP BY列必须建立索引

sql复制-- 对于 GROUP BY product_id
CREATE INDEX idx_products ON sales(product_id);

复合索引顺序很重要

sql复制-- 对于 WHERE date > ? GROUP BY category
CREATE INDEX idx_sales_date_category ON sales(date, category);

覆盖索引可以避免回表

sql复制-- 对于 SELECT category, COUNT(*) FROM sales GROUP BY category
CREATE INDEX idx_sales_category ON sales(category);

5.2 大数据量优化技巧

使用近似聚合

sql复制-- 使用HyperLogLog估算唯一值计数
SELECT APPROX_COUNT_DISTINCT(user_id) FROM large_table;

分阶段聚合

sql复制-- 先按天聚合，再按月聚合
WITH daily AS (
    SELECT date_trunc('day', ts) AS day, COUNT(*) AS cnt
    FROM events
    GROUP BY 1
)
SELECT date_trunc('month', day) AS month, SUM(cnt)
FROM daily
GROUP BY 1;

使用物化视图

sql复制CREATE MATERIALIZED VIEW sales_summary AS
SELECT product_id, SUM(amount) AS total_sales
FROM sales
GROUP BY product_id;

-- 定期刷新
REFRESH MATERIALIZED VIEW sales_summary;

5.3 执行计划分析

理解EXPLAIN输出是关键：

sql复制EXPLAIN ANALYZE
SELECT department, AVG(salary)
FROM employees
WHERE hire_date > '2020-01-01'
GROUP BY department;

重点关注：

是否使用了正确的索引（Index Scan）
是否有昂贵的排序操作（Sort）
预估行数和实际行数是否匹配
是否有全表扫描（Seq Scan）

6. 高级聚合模式

6.1 窗口函数与聚合

窗口函数可以增强聚合能力：

sql复制SELECT 
    department,
    employee_id,
    salary,
    AVG(salary) OVER (PARTITION BY department) AS avg_salary,
    salary - AVG(salary) OVER (PARTITION BY department) AS diff_from_avg
FROM employees;

6.2 多层次聚合

使用GROUPING SETS、CUBE和ROLLUP：

sql复制-- 多层次聚合
SELECT 
    COALESCE(department, 'All') AS department,
    COALESCE(team, 'All') AS team,
    COUNT(*) AS employee_count
FROM employees
GROUP BY GROUPING SETS (
    (department, team),
    (department),
    ()
);

-- 同比环比计算
WITH monthly_sales AS (
    SELECT 
        date_trunc('month', order_date) AS month,
        SUM(amount) AS sales
    FROM orders
    GROUP BY 1
)
SELECT 
    month,
    sales,
    LAG(sales, 1) OVER (ORDER BY month) AS prev_month,
    sales - LAG(sales, 1) OVER (ORDER BY month) AS mom_growth
FROM monthly_sales;

6.3 条件聚合

使用FILTER子句：

sql复制SELECT
    product_id,
    COUNT(*) AS total_orders,
    COUNT(*) FILTER (WHERE rating = 5) AS five_star_orders,
    COUNT(*) FILTER (WHERE rating < 3) AS low_rating_orders
FROM orders
GROUP BY product_id;

7. 实际业务场景应用

7.1 电商业务分析

用户购买行为分析：

sql复制SELECT
    user_id,
    COUNT(DISTINCT order_id) AS order_count,
    SUM(amount) AS total_spend,
    AVG(amount) AS avg_order_value,
    MIN(created_at) AS first_order_date,
    MAX(created_at) AS last_order_date,
    EXTRACT(DAY FROM MAX(created_at) - MIN(created_at)) AS customer_lifetime_days
FROM orders
WHERE status = 'completed'
GROUP BY user_id
HAVING COUNT(DISTINCT order_id) > 1
ORDER BY total_spend DESC;

商品销售分析：

sql复制SELECT
    p.product_id,
    p.product_name,
    COUNT(DISTINCT o.user_id) AS customer_count,
    SUM(oi.quantity) AS total_units_sold,
    SUM(oi.quantity * oi.price) AS total_revenue,
    SUM(oi.quantity * oi.price) / SUM(oi.quantity) AS avg_selling_price,
    COUNT(DISTINCT CASE WHEN o.created_at >= CURRENT_DATE - INTERVAL '30 days' 
                        THEN o.order_id END) AS recent_order_count
FROM products p
JOIN order_items oi ON p.product_id = oi.product_id
JOIN orders o ON oi.order_id = o.order_id
WHERE o.status = 'completed'
GROUP BY p.product_id, p.product_name
ORDER BY total_revenue DESC;

7.2 用户活跃度分析

sql复制WITH user_activity AS (
    SELECT
        user_id,
        DATE_TRUNC('day', event_time) AS activity_date,
        COUNT(DISTINCT event_id) AS event_count
    FROM user_events
    WHERE event_time >= CURRENT_DATE - INTERVAL '90 days'
    GROUP BY user_id, DATE_TRUNC('day', event_time)
),
activity_stats AS (
    SELECT
        user_id,
        COUNT(*) AS active_days,
        SUM(event_count) AS total_events,
        MIN(activity_date) AS first_active_day,
        MAX(activity_date) AS last_active_day
    FROM user_activity
    GROUP BY user_id
)
SELECT
    user_id,
    active_days,
    total_events,
    active_days / EXTRACT(DAY FROM (MAX(last_active_day) OVER () - MIN(first_active_day) OVER ())) AS activity_ratio,
    CASE 
        WHEN active_days >= 20 THEN '高频用户'
        WHEN active_days >= 10 THEN '中频用户'
        WHEN active_days >= 1 THEN '低频用户'
        ELSE '流失用户'
    END AS user_segment
FROM activity_stats;

8. 面试常见问题解析

8.1 高频面试题

如何计算留存率？

sql复制WITH cohort AS (
    SELECT 
        user_id,
        DATE_TRUNC('week', first_purchase_date) AS signup_week
    FROM users
),
activity AS (
    SELECT
        c.signup_week,
        DATE_TRUNC('week', o.order_date) AS activity_week,
        COUNT(DISTINCT c.user_id) AS users
    FROM cohort c
    LEFT JOIN orders o ON c.user_id = o.user_id
       AND o.order_date BETWEEN c.first_purchase_date 
       AND c.first_purchase_date + INTERVAL '7 days'
    GROUP BY 1, 2
)
SELECT
    signup_week,
    MAX(CASE WHEN activity_week = signup_week THEN users END) AS week_0,
    MAX(CASE WHEN activity_week = signup_week + INTERVAL '1 week' THEN users END) AS week_1,
    MAX(CASE WHEN activity_week = signup_week + INTERVAL '2 weeks' THEN users END) AS week_2
FROM activity
GROUP BY signup_week;

如何找出连续登录的用户？

sql复制WITH login_dates AS (
    SELECT 
        user_id,
        login_date,
        login_date - ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY login_date) AS grp
    FROM user_logins
)
SELECT
    user_id,
    MIN(login_date) AS start_date,
    MAX(login_date) AS end_date,
    COUNT(*) AS consecutive_days
FROM login_dates
GROUP BY user_id, grp
HAVING COUNT(*) >= 3
ORDER BY user_id, start_date;

8.2 性能调优问题

问题： 有一个大表的聚合查询很慢，如何优化？

解决方案：

检查GROUP BY列是否有索引
考虑使用物化视图预计算结果
对于精确计数，可以使用增量计算
对于大表，考虑使用分区表
评估是否可以使用采样近似计算

sql复制-- 分区表示例
CREATE TABLE sales (
    sale_id BIGINT,
    product_id INT,
    sale_date DATE,
    amount DECIMAL(10,2)
) PARTITION BY RANGE (sale_date);

-- 创建每月分区
CREATE TABLE sales_202301 PARTITION OF sales
    FOR VALUES FROM ('2023-01-01') TO ('2023-02-01');

8.3 复杂业务逻辑实现

问题： 如何计算用户购买路径的转化率？

sql复制WITH funnel AS (
    SELECT
        user_id,
        MAX(CASE WHEN event_type = 'view' THEN 1 ELSE 0 END) AS viewed_product,
        MAX(CASE WHEN event_type = 'add_to_cart' THEN 1 ELSE 0 END) AS added_to_cart,
        MAX(CASE WHEN event_type = 'checkout' THEN 1 ELSE 0 END) AS checked_out
    FROM user_events
    WHERE event_time BETWEEN '2023-01-01' AND '2023-01-31'
    GROUP BY user_id
)
SELECT
    COUNT(*) AS total_users,
    SUM(viewed_product) AS viewers,
    SUM(added_to_cart) AS cart_adders,
    SUM(checked_out) AS purchasers,
    ROUND(100.0 * SUM(added_to_cart) / NULLIF(SUM(viewed_product), 0), 2) AS view_to_cart_rate,
    ROUND(100.0 * SUM(checked_out) / NULLIF(SUM(added_to_cart), 0), 2) AS cart_to_purchase_rate
FROM funnel;

9. 最佳实践与经验分享

9.1 代码规范建议

格式化标准
- GROUP BY列与SELECT列顺序一致
- 聚合函数换行对齐
- 复杂表达式使用括号明确优先级
命名规范
- 别名使用有意义的名称
- 临时表使用CTE而非子查询
- 使用AS明确列别名
注释规范
- 复杂业务逻辑添加注释
- 非常规操作说明原因
- 临时解决方案标注TODO

9.2 常见陷阱

NULL值处理
- COUNT(*) vs COUNT(column)
- SUM(NULL)返回NULL而非0
- AVG忽略NULL值
分组陷阱
- SELECT非聚合列必须出现在GROUP BY
- HAVING中使用聚合函数而非WHERE
- GROUP BY表达式与SELECT表达式一致
性能问题
- 大表GROUP BY无索引
- 多列GROUP BY导致组合爆炸
- 聚合后排序未使用LIMIT

9.3 调试技巧

逐步验证法

sql复制-- 1. 先验证基础数据
SELECT * FROM table WHERE condition LIMIT 10;

-- 2. 验证聚合前的数据
SELECT column, COUNT(*) 
FROM table 
WHERE condition
GROUP BY column;

-- 3. 完整查询
SELECT column, AVG(value)
FROM table
WHERE condition
GROUP BY column;

使用CTE分解复杂查询

sql复制WITH base_data AS (
    SELECT * FROM table WHERE condition
),
aggregated AS (
    SELECT key, SUM(value) AS total
    FROM base_data
    GROUP BY key
)
SELECT * FROM aggregated ORDER BY total DESC;

比较不同方法的执行计划

sql复制EXPLAIN ANALYZE SELECT... -- 版本A
EXPLAIN ANALYZE SELECT... -- 版本B

10. 实战案例：电商数据分析系统

10.1 数据模型设计

sql复制-- 产品维度表
CREATE TABLE dim_products (
    product_key INT PRIMARY KEY,
    product_id INT,
    product_name VARCHAR(255),
    category VARCHAR(100),
    price DECIMAL(10,2),
    effective_date DATE,
    expiration_date DATE,
    current_flag BOOLEAN
);

-- 销售事实表
CREATE TABLE fact_sales (
    sale_id BIGINT PRIMARY KEY,
    product_key INT REFERENCES dim_products(product_key),
    date_key INT REFERENCES dim_dates(date_key),
    customer_key INT REFERENCES dim_customers(customer_key),
    quantity INT,
    amount DECIMAL(12,2),
    discount DECIMAL(10,2),
    net_amount DECIMAL(12,2)
);

-- 日期维度表
CREATE TABLE dim_dates (
    date_key INT PRIMARY KEY,
    full_date DATE,
    day_of_week INT,
    month INT,
    quarter INT,
    year INT,
    is_weekend BOOLEAN,
    is_holiday BOOLEAN
);

10.2 核心分析查询

销售趋势分析：

sql复制WITH monthly_sales AS (
    SELECT
        d.year,
        d.month,
        SUM(f.amount) AS gross_sales,
        SUM(f.net_amount) AS net_sales,
        COUNT(DISTINCT f.customer_key) AS customers,
        SUM(f.quantity) AS units
    FROM fact_sales f
    JOIN dim_dates d ON f.date_key = d.date_key
    WHERE d.full_date BETWEEN '2022-01-01' AND '2022-12-31'
    GROUP BY d.year, d.month
)
SELECT
    year,
    month,
    gross_sales,
    net_sales,
    gross_sales - net_sales AS discount_amount,
    units,
    gross_sales / NULLIF(units, 0) AS avg_unit_price,
    gross_sales / NULLIF(customers, 0) AS avg_order_value,
    gross_sales - LAG(gross_sales) OVER (ORDER BY year, month) AS mom_growth
FROM monthly_sales
ORDER BY year, month;

产品关联分析：

sql复制SELECT
    p1.product_name AS product_a,
    p2.product_name AS product_b,
    COUNT(DISTINCT s1.customer_key) AS co_purchase_count
FROM fact_sales s1
JOIN fact_sales s2 ON s1.customer_key = s2.customer_key
    AND DATE_TRUNC('day', s1.sale_date) = DATE_TRUNC('day', s2.sale_date)
    AND s1.product_key < s2.product_key
JOIN dim_products p1 ON s1.product_key = p1.product_key
JOIN dim_products p2 ON s2.product_key = p2.product_key
GROUP BY p1.product_name, p2.product_name
HAVING COUNT(DISTINCT s1.customer_key) > 10
ORDER BY co_purchase_count DESC
LIMIT 20;

10.3 自动化报表实现

sql复制-- 创建物化视图
CREATE MATERIALIZED VIEW mv_daily_sales AS
SELECT
    d.full_date,
    p.category,
    COUNT(DISTINCT s.customer_key) AS customers,
    SUM(s.quantity) AS units,
    SUM(s.amount) AS gross_sales,
    SUM(s.net_amount) AS net_sales
FROM fact_sales s
JOIN dim_dates d ON s.date_key = d.date_key
JOIN dim_products p ON s.product_key = p.product_key
GROUP BY d.full_date, p.category;

-- 创建刷新存储过程
CREATE OR REPLACE PROCEDURE refresh_sales_views()
LANGUAGE plpgsql
AS $$
BEGIN
    REFRESH MATERIALIZED VIEW mv_daily_sales;
    REFRESH MATERIALIZED VIEW mv_product_performance;
    REFRESH MATERIALIZED VIEW mv_customer_segments;
END;
$$;

-- 设置定时任务
-- 在Linux crontab中添加：
-- 0 2 * * * psql -U user -d dbname -c "CALL refresh_sales_views()"