1. Introduction

In this article we will discuss over() function in SQL. over() is used in conjunction with other functions and mostly it means scatter the previous function's result over the original table rows.

 

2. The Data

We will use classical dataset flights.csv. It is each month passengers number between 1949 and 1960.

First we will copy the dataset into /tmp folder, which postgresql has permission to read.

$ cp flights.csv /tmp

Second we will create a new table in SQL and import data with copy.

create table flights (
	year int,
	month varchar,
	passengers int
);

copy flights
from '/tmp/flights.csv'
with csv header;

　　

3. Ranking

To calculate the rank of a numeric column, we have rank() in SQL. But it has to work together with over().

We can use “order by” clause to specify ranking order.

select *, rank() over(order by passengers desc)
from flights;

We can use “partition by” clause to limit the calculation range only within each year.

Most of the time this is more reasonable method of ranking real world case.

select *, rank() over(partition by year order by passengers desc)
from flights;

There are also other kinds of ranking functions but that is another story. One common thing is they all have to work together with over().

select *, dense_rank() over(partition by year order by passengers desc)
from flights;

select *, row_number() over(partition by year order by passengers desc)
from flights;

 

4. Aggregation　　

Normally an aggregate function will has a result with less row than the original table. But using over() we can scatter the result back to original table rows.

select *, sum(passengers) over()
from flights;

Using this trick we can do calculation between columns more easily.

select *, passengers / avg_yearly as floating_points
from (
	select *, avg(passengers) over(partition by year) as avg_yearly
	from flights
) as avg_cal
;

　　

5. Cumsum(Cumulative Summary)

If we want to do cumsum in SQL, we will still have to use sum() with over(). With the “order by” clause we can specify the cumulating order. And with the “partition by” clause we can limit the culmulating calculation only within a range.

select *, sum(passengers) over(partition by year order by month)
from flights;

 

6. Moving Average(Rolling Mean)　　

Last topic today of over() is moving average. We can use a special grammar “rows between ... and ... row” to specify summary range. This can be used as moving average.

select *, avg(passengers) over(partition by year rows between 2 preceding and current row)
from flights;