Introduction to Pandas · Data Analysis with Python

Before diving in, let's quickly review two Python concepts this lesson depends on. If you already know these, feel free to skim.

Python Lists

A list is a collection of values stored in order. Think of it like a numbered shopping list.

fruits = ["apple", "banana", "cherry"]
print(fruits[0])   # Access first item by its index number

Expected Output:

apple

Each item has an index · a number starting from 0 that tells Python where in the list that item lives.

Python Dictionaries

A dictionary stores data as key-value pairs · like a word and its definition in a real dictionary.

person = {"name": "Amara", "age": 22, "city": "Lagos"}
print(person["name"])

Expected Output:

Amara

You use the key (the word on the left) to look up its value (the data on the right). Pandas uses dictionaries heavily when creating tables of data.

Python import Statement

When you want to use an external tool or library in Python, you bring it in with import.

import math
print(math.sqrt(25))

Expected Output:

5.0

You will use import to bring Pandas into every program you write in this lesson.

The Big Picture · Why Does Pandas Exist?

Imagine you work for a hospital, a bank, or a university. Every day, thousands of records are collected: patient readings, transactions, exam scores. You need to answer questions like:

• What is the average score across all students?
• Which patient has the highest blood pressure?
• On which day did the store earn the most money?

Before Pandas, answering these questions required writing long, complex Python code just to organise the data, let alone analyse it.

Pandas was created in 2008 by Wes McKinney to solve exactly this problem. The name stands for "Panel Data" and "Python Data Analysis" · two hints at what it does.

What Pandas Actually Does

Pandas is a Python library. A library is simply a collection of pre-written code that someone else (or a team) built and packaged for you to use freely. When you use Pandas, you are borrowing thousands of hours of work to do powerful things in just a few lines.

With Pandas you can:

• Analyse big datasets · find averages, maximums, minimums
• Clean messy data · fix errors, remove blanks, correct wrong values
• Explore data · look at its shape, types, and patterns
• Manipulate data · sort, filter, combine, and reshape tables
• Load data from external files · CSV, JSON, Excel, and more

💡 Real-world analogy: Think of Pandas as a supercharged spreadsheet that you control with Python code. Excel can handle perhaps a few hundred thousand rows before slowing down. Pandas can handle millions of rows efficiently.

Where Is Pandas Stored?

Pandas is an open-source project. Its code lives at https://github.com/pandas-dev/pandas where developers from all over the world contribute improvements. GitHub is a platform where many people can work on the same codebase at once · think of it as Google Docs for code.

What Fields Use Pandas?

Pandas is used in virtually every field that works with data:

• Data Science and Machine Learning · preparing datasets for AI models
• Finance and Banking · analysing stock prices, transactions, risk
• Healthcare · processing patient records and clinical trial data
• Engineering · analysing sensor readings and measurements
• Education · processing exam results and student performance
• Business · sales analysis, customer behaviour, inventory tracking

Step 1: Install Pandas

Before you can use Pandas, you must install it. Pandas is not built into Python by default, so you need to add it using PIP · Python's package manager.

Open your terminal (also called the command prompt or shell) and type:

pip install pandas

What this command does:

• pip · this is Python's install helper tool
• install · tells pip to download and install a package
• pandas · the name of the package to install

If the installation is successful, you will see output ending with something like:

Successfully installed pandas-2.x.x

💡 Tip: If you are using an environment like Anaconda, Spyder, or Google Colab, Pandas is usually already pre-installed. You can skip the installation step and go straight to importing.

Step 2: Import Pandas

Once Pandas is installed, you bring it into your Python script using the import keyword. You do this at the very top of every file where you want to use Pandas.

The full import (less common):

import pandas

print(pandas.__version__)

Expected Output:

2.x.x

The standard import with an alias (used everywhere):

import pandas as pd

print(pd.__version__)

Expected Output:

2.x.x

Understanding the as pd Alias

An alias is a shorter nickname for something. Instead of typing pandas every time you want to use the library, you give it the short name pd. This is a universal convention · every Pandas programmer in the world uses import pandas as pd. You will see pd used in all examples from this point forward.

Think of it like this: instead of saying "please pass me the television remote control", you just say "remote". Same object, shorter name.

Verifying the version:

import pandas as pd

print(pd.__version__)

The __version__ attribute (the double underscores mean it is a special built-in attribute) stores the version string of the installed library. Checking this is a simple way to confirm Pandas loaded correctly.

What Is a Series?

A Pandas Series is a one-dimensional array · meaning it is a single column of data. You can think of it as a list, but smarter.

💡 Analogy: If a DataFrame is a full spreadsheet, a Series is one single column of that spreadsheet. For example, the "Temperature" column in a weather table, or the "Score" column in a student grade table.

A Series can hold any type of data: numbers, text, dates, True/False values.

Creating a Simple Series from a List

import pandas as pd

a = [1, 7, 2]

myvar = pd.Series(a)

print(myvar)

Expected Output:

0    1
1    7
2    2
dtype: int64

Breaking down this output line by line:

• The left column (0, 1, 2) shows the index · the label for each position, automatically assigned starting from 0
• The right column (1, 7, 2) shows the values · the actual data you provided
• dtype: int64 · this tells you the data type. int64 means 64-bit integer (whole numbers). Pandas automatically detects what type your data is.

🤔 Thinking prompt: What happens if you change a = [1, 7, 2] to a = [10, 20, 30]? Try it · does the index change? Why or why not?

Accessing Values by Index

Once a Series is created, you can retrieve any value using its index number inside square brackets [].

import pandas as pd

a = [1, 7, 2]
myvar = pd.Series(a)

print(myvar[0])   # First value (index 0)
print(myvar[1])   # Second value (index 1)
print(myvar[2])   # Third value (index 2)

Expected Output:

1
7
2

The index always starts at 0, not 1. This is called zero-based indexing and is standard in Python.

Creating a Series with Custom Labels

By default, Pandas numbers the index 0, 1, 2, and so on. But you can give your own meaningful names to the index positions using the index argument.

import pandas as pd

a = [1, 7, 2]

myvar = pd.Series(a, index = ["x", "y", "z"])

print(myvar)

Expected Output:

x    1
y    7
z    2
dtype: int64

Now the index labels are "x", "y", and "z" instead of numbers. This makes it easier to understand what each value represents.

Accessing by custom label:

print(myvar["y"])

Expected Output:

7

You can use either the label ("y") or the numeric position ([1]) to access the value · they both work!

Creating a Series from a Dictionary

A very common way to create a Series is from a Python dictionary, because dictionaries already have the key-value structure that maps perfectly to index-value pairs.

import pandas as pd

calories = {"day1": 420, "day2": 380, "day3": 390}

myvar = pd.Series(calories)

print(myvar)

Expected Output:

day1    420
day2    380
day3    390
dtype: int64

What happened here?

• The dictionary keys ("day1", "day2", "day3") automatically became the index labels
• The dictionary values (420, 380, 390) became the Series values

This is extremely useful when you have labelled data · like calorie counts per day, or temperatures per city.

Selecting Only Some Items from a Dictionary

Sometimes you have a dictionary with many entries but only want to include some of them in your Series. Use the index argument to specify which keys to include:

import pandas as pd

calories = {"day1": 420, "day2": 380, "day3": 390}

myvar = pd.Series(calories, index = ["day1", "day2"])

print(myvar)

Expected Output:

day1    420
day2    380
dtype: int64

Only day1 and day2 were included. day3 was ignored because it was not listed in the index argument.

🤔 Thinking prompt: What do you think would happen if you listed a key in index that does NOT exist in the dictionary? Try index = ["day1", "day5"]. What does Pandas return for day5?

What Is a DataFrame?

If a Series is a single column, a DataFrame is the entire table · rows AND columns.

A Pandas DataFrame is a two-dimensional (2D) data structure, similar to a spreadsheet or a SQL table. It has:

• Rows · each row represents one record (e.g., one student, one day's workout, one product)
• Columns · each column represents one attribute or measurement (e.g., name, score, price)
• Index · labels for each row (automatically numbered 0, 1, 2... unless you name them)

💡 Analogy: Think of a DataFrame as an Excel sheet. Each column is a Series. Multiple Series stacked side by side form a DataFrame.

Creating a Simple DataFrame

You create a DataFrame by passing a Python dictionary to pd.DataFrame(). Each key in the dictionary becomes a column name, and each list of values becomes that column's data.

import pandas as pd

data = {
  "calories": [420, 380, 390],
  "duration": [50, 40, 45]
}

df = pd.DataFrame(data)

print(df)

Expected Output:

   calories  duration
0       420        50
1       380        40
2       390        45

Explaining every part of this output:

• The top row (calories duration) shows the column headers · one for each key in the dictionary
• The leftmost column (0, 1, 2) shows the row index · automatically assigned, starting at 0
• Each row of numbers represents one record. Row 0 says "this workout burned 420 calories and lasted 50 minutes"

By convention, the variable holding a DataFrame is often called df. This is just a short, recognisable name · like how we use x for unknowns in maths.

Locating a Specific Row Using .loc

Once you have a DataFrame, you often need to look at one specific row. Pandas provides the .loc attribute (short for "locate") for this purpose.

Access a single row by index number:

import pandas as pd

data = {
  "calories": [420, 380, 390],
  "duration": [50, 40, 45]
}

df = pd.DataFrame(data)

print(df.loc[0])

Expected Output:

calories    420
duration     50
Name: 0, dtype: int64

Notice: when you locate a single row, the result is displayed vertically and is actually a Pandas Series! The column names become the index labels, and the values in that row become the values.

Access multiple rows by passing a list of indexes:

print(df.loc[[0, 1]])

Expected Output:

   calories  duration
0       420        50
1       380        40

When you pass a list [0, 1] to .loc, the result is a DataFrame (a table with multiple rows). Notice the double square brackets · [[0, 1]] not [0, 1].

💡 Key rule: .loc[0] → returns a Series (one row). .loc[[0, 1]] → returns a DataFrame (multiple rows).

Creating a DataFrame with Named Row Indexes

By default, row indexes are numbers (0, 1, 2...). You can give rows meaningful names using the index parameter:

import pandas as pd

data = {
  "calories": [420, 380, 390],
  "duration": [50, 40, 45]
}

df = pd.DataFrame(data, index = ["day1", "day2", "day3"])

print(df)

Expected Output:

      calories  duration
day1       420        50
day2       380        40
day3       390        45

Now each row is labelled with a meaningful name. This is very useful for time-series data (data measured over days, months, years).

Accessing a named row:

print(df.loc["day2"])

Expected Output:

calories    380
duration     40
Name: day2, dtype: int64

You used the string "day2" as the row label, and .loc found it instantly.

🤔 Thinking prompt: Can you add a third column called "heartrate" with values [120, 110, 115] to the dictionary and recreate the DataFrame? What does the output look like?

Loading Data from Files into a DataFrame

In real work, you will rarely type all your data manually. You will load it from external files. The most common types are CSV and JSON.

import pandas as pd

df = pd.read_csv('data.csv')

print(df)

This loads an entire file of data into a DataFrame with one line of code. You will learn the full details of this in the next sections.

What Is a CSV File?

A CSV file (Comma-Separated Values) is a plain text file that stores tabular data. Each row in the file is one record, and the values within each row are separated by commas.

Here is what a simple CSV file looks like when you open it in a text editor:

Duration,Pulse,Maxpulse,Calories
60,110,130,409
60,117,145,479
60,103,135,340
45,109,175,282
45,117,148,406
60,102,127,300

The first line is the header row · it contains the column names. Every line after that is one row of data. CSV files are extremely common because they are simple, human-readable, and supported by almost every tool including Excel, Python, and databases.

💡 Real-world use: CSV files are how many banks export transaction histories, how hospitals share patient records between systems, and how weather services distribute climate data.

Reading a CSV File with Pandas

Pandas has a built-in function called pd.read_csv() that reads a CSV file and converts it into a DataFrame automatically.

import pandas as pd

df = pd.read_csv('data.csv')

print(df.to_string())

What each line does:

• import pandas as pd · loads the Pandas library with the short name pd
• pd.read_csv('data.csv') · reads the file named data.csv and builds a DataFrame from it, stored in df
• df.to_string() · converts the entire DataFrame to a string for printing, so you see every row

Expected Output (abbreviated):

   Duration  Pulse  Maxpulse  Calories
0        60    110       130     409.1
1        60    117       145     479.0
2        60    103       135     340.0
3        45    109       175     282.4
4        45    117       148     406.0
...

💡 Tip: Without .to_string(), if the DataFrame has many rows, Pandas only prints the first 5 and the last 5 rows by default, with ... in between. Using to_string() forces it to display everything.

Printing Without .to_string()

If you just do print(df) on a large dataset:

import pandas as pd

df = pd.read_csv('data.csv')

print(df)

Expected Output (for a large file):

     Duration  Pulse  Maxpulse  Calories
0          60    110       130     409.1
1          60    117       145     479.0
2          60    103       135     340.0
3          45    109       175     282.4
4          45    117       148     406.0
..        ...    ...       ...       ...
164        60    105       140     290.8
165        60    110       145     300.0
...
[169 rows x 5 columns]

Pandas truncates the output to be readable. It shows 5 rows from the top and 5 from the bottom, then shows you the total number of rows and columns at the bottom.

Checking and Changing the Maximum Display Rows

Pandas has a setting that controls how many rows are shown when you print a DataFrame. You can check it like this:

import pandas as pd

print(pd.options.display.max_rows)

Expected Output:

60

This means: if your DataFrame has more than 60 rows, Pandas will only show the first and last 5 when you print(df).

Changing the maximum rows:

import pandas as pd

pd.options.display.max_rows = 9999

df = pd.read_csv('data.csv')

print(df)

By setting max_rows to 9999, you are telling Pandas to show up to 9,999 rows without truncating. This is useful when you want to inspect the full dataset.

Why does this setting exist? Because some real-world datasets have millions of rows. Printing all of them to the terminal would be both slow and useless. Pandas defaults to showing a manageable summary.

What Is JSON?

JSON stands for JavaScript Object Notation. Despite the name mentioning JavaScript, JSON is a universal data format used everywhere · web APIs, databases, configuration files, and data science pipelines.

JSON uses a structure that looks very similar to a Python dictionary:

{
  "Duration": {"0": 60, "1": 60, "2": 60},
  "Pulse":    {"0": 110, "1": 117, "2": 103},
  "Calories": {"0": 409, "1": 479, "2": 340}
}

The outer keys ("Duration", "Pulse", "Calories") are column names. The inner keys ("0", "1", "2") are row indexes. The inner values are the data.

💡 Key insight: JSON = Python Dictionary (in terms of structure). This is why Pandas can load both with ease.

Reading a JSON File

Just like CSV, Pandas has a dedicated function: pd.read_json().

import pandas as pd

df = pd.read_json('data.json')

print(df.to_string())

Expected Output:

   Duration  Pulse  Maxpulse  Calories
0        60    110       130     409.1
1        60    117       145     479.0
2        60    103       135     340.0
...

The result is identical to reading the CSV · a proper DataFrame with column headers and row indexes. Pandas handles the conversion for you.

Loading a Python Dictionary Directly as JSON

Since JSON and Python dictionaries share the same structure, you can also pass a Python dictionary directly to pd.DataFrame() to simulate reading JSON data without needing a file:

import pandas as pd

data = {
  "Duration": {
    "0": 60,
    "1": 60,
    "2": 60,
    "3": 45,
    "4": 45,
    "5": 60
  },
  "Pulse": {
    "0": 110,
    "1": 117,
    "2": 103,
    "3": 109,
    "4": 117,
    "5": 102
  },
  "Maxpulse": {
    "0": 130,
    "1": 145,
    "2": 135,
    "3": 175,
    "4": 148,
    "5": 127
  },
  "Calories": {
    "0": 409,
    "1": 479,
    "2": 340,
    "3": 282,
    "4": 406,
    "5": 300
  }
}

df = pd.DataFrame(data)

print(df)

Expected Output:

   Duration  Pulse  Maxpulse  Calories
0        60    110       130       409
1        60    117       145       479
2        60    103       135       340
3        45    109       175       282
4        45    117       148       406
5        60    102       127       300

What happened here, step by step:

1. We defined a nested dictionary where outer keys are column names and inner keys are row index labels
2. pd.DataFrame(data) read that dictionary and converted it into a DataFrame
3. Pandas automatically aligned all the values by their matching index keys

This approach is useful when your data comes from an API (a web service that returns JSON) and you want to quickly turn it into a DataFrame for analysis.

CSV vs JSON · A Quick Comparison

Both formats are extremely common. A data analyst needs to be comfortable with both.

Mistake 1: Forgetting import pandas as pd

# WRONG — Pandas not imported
df = pd.DataFrame({"a": [1, 2, 3]})

Error:

NameError: name 'pd' is not defined

Fix: Always start your file with import pandas as pd.

# CORRECT
import pandas as pd
df = pd.DataFrame({"a": [1, 2, 3]})

Mistake 2: Single Brackets vs Double Brackets with .loc

import pandas as pd
data = {"calories": [420, 380, 390], "duration": [50, 40, 45]}
df = pd.DataFrame(data)

# This returns a Series (one row)
print(df.loc[0])

# This returns a DataFrame (still one row but in table format)
print(df.loc[[0]])

Output comparison:

df.loc[0] → Series (vertical display) df.loc[[0]] → DataFrame (horizontal table display)

Use double brackets when you want to keep the table structure intact.

Mistake 3: Mixing Up List and Dictionary Formats

# WRONG — passing a flat list to pd.DataFrame creates an error
import pandas as pd
df = pd.DataFrame([1, 2, 3])   # This actually works but creates something unexpected
print(df)

Output:

   0
0  1
1  2
2  3

This creates a DataFrame with one column (named 0) and the numbers as rows. This is rarely what you intend. The proper way to create a meaningful DataFrame is using a dictionary:

# CORRECT
import pandas as pd
df = pd.DataFrame({"values": [1, 2, 3]})
print(df)

Output:

   values
0       1
1       2
2       3

Mistake 4: Wrong File Path for CSV/JSON

import pandas as pd
df = pd.read_csv('myfile.csv')

Error:

FileNotFoundError: [Errno 2] No such file or directory: 'myfile.csv'

Causes and fixes:

• The file doesn't exist → create or download the file
• The file is in a different folder → provide the full path, e.g., pd.read_csv('/home/user/data/myfile.csv')
• Wrong file extension → check you're not accidentally trying to read a .txt file as .csv

Mistake 5: Accessing an Index That Doesn't Exist

import pandas as pd
a = [10, 20, 30]
s = pd.Series(a)
print(s[5])   # There is no index 5 — only 0, 1, 2 exist

Error:

KeyError: 5

Fix: Always check the length of your Series first with len(s) or print the Series to see its valid indexes before accessing.

Mistake 6: Confusing .loc Label Access with .iloc Position Access

Pandas has two ways to access rows:

• .loc["label"] · access by label (the name you gave the index)
• .iloc[2] · access by position (the numeric position, always starting from 0)

If you have named indexes and you try to use a number with .loc, you might get unexpected results or errors.

import pandas as pd
data = {"value": [100, 200, 300]}
df = pd.DataFrame(data, index=["a", "b", "c"])

# Using .loc with label
print(df.loc["b"])     # Correct — uses label "b"

# Using .iloc with position
print(df.iloc[1])      # Correct — uses position 1 (which is "b")

Both give the same result here, but always know which one you need.