Chapter 3
Load and Inspect a Table
Meet the DataFrame
The Chapter 1 project answered its question in about seventy lines, and most of them read rows one at a time. Chapter 2 showed that a whole column of numbers can be one
A table in one call
Here is how you load the readings file with pandas. The first line imports it under the short name pd, which almost all pandas code uses, just as np:
import pandas as pd
readings = pd.read_csv("data/readings.csv")
print(readings.shape)
print(list(readings.columns))
pd.read_csv opens the file, reads the header, splits every line at the commas, and returns one DataFrame holding all of it. There is no with block, no csv.DictReader and no open() in Python in Practice, and pd.read_csv also accepts a pathlib.Path.
On data/readings.csv, the two print calls show:
(1079, 6)
['station_id', 'date', 'temp_min_c', 'temp_max_c', 'rain_mm', 'battery_low']
readings.shape works like the .shape of a NumPy array: a pair of numbers, rows first, then columns. The file has 1,079 rows of readings and 6 columns. readings.columns holds the column names, in the order of the file’s header, and list(...) turns them into an ordinary Python list() is a built-in list("ST01"), for example, is ['S', 'T', '0', '1'], because a
The block above has no Run button. The Run buttons in a lesson run in your browser, which cannot see the project’s files, so any example that reads data/readings.csv is shown for reading only. To experiment in the browser, you build a small table in the code itself.
Build a small table
pd.DataFrame(...) builds a table from a
Printing the table shows it as a grid:
station_id rain_mm
0 ST01 0.0
1 ST02 6.0
2 ST01 5.8
The two keys became the two column names across the top. Each list became a column, and all the lists have the same length, three, because every row needs one value in each column. table.shape is (3, 2): three rows, two columns.
The numbers down the left side, 0, 1 and 2, are not part of your data. They are the table’s pd.read_csv and pd.DataFrame both number the rows from 0 unless you ask for something else, so the readings table is labeled 0 to 1078. table.index holds those labels, and list(table.index) prints them as [0, 1, 2].
Change the rain_mm list to hold four values and run the block again. pandas refuses with a
What does this print?
import pandas as pd
table = pd.DataFrame({
"station_id": ["ST03", "ST03", "ST04", "ST04"],
"temp_min_c": [-1.7, -4.4, -2.9, -3.7],
"rain_mm": [0.0, 0.0, 0.0, 5.8],
})
print(table.shape)Columns are arrays with names
A DataFrame is a set of columns that share one index. Each column works like a Chapter 2 array: all its values have one dtype, and arithmetic and summaries work on the whole column at once. A column of numbers is stored as a NumPy array underneath. What the table adds is a name on top of each column and a label beside each row.
Because each column has its own dtype, one table can mix kinds of values. In the example, station_id holds text and rain_mm holds decimal numbers. A single NumPy array could not hold both kinds and still do arithmetic, but a table of two arrays can.
So the rows of the readings table are the records you read one at a time in Chapter 1, and the columns are what Chapter 2 computed with. The next lesson takes one column out of the table and computes with it.
| Chapter 1 | Chapter 2 | Chapter 3 |
|---|---|---|
a row as a dictionary from csv.DictReader | a column as a NumPy array | a whole table as one DataFrame |
| a loop visits every row | one | a column name picks the column |
Which statement about this table is true?
table = pd.DataFrame({"station_id": ["ST01", "ST02"], "rain_mm": [0.0, 6.0]})
Nothing has been computed yet: the readings are in memory as one load.py, and write your first functions that work with that object.