Pandas is a powerful data manipulation and analysis library for Python. It provides data structures like series and dataframes to effectively easily clean, transform, and analyze large datasets and integrates seamlessly with other python libraries, such as numPy and matplotlib. It offers powerful functions for data transformation, aggregation, and visualization, which are crucial for effective analysis.
Pandas revolves around two primary data structures: series (1D) for single columns and dataframe (2D) for tabular data, enabling efficient data manipulation.
Section 1: Installing and Importing Pandas
To install Pandas, run the following command:
pip install pandas
To use Pandas in your code, import it with:
import pandas as pd
This imports the Pandas library and gives it the alias pd for convenience.
Section 2: Creating DataFrames
- DataFrame is a two-dimensional table-like data structure with labeled rows and columns, where each column can have a different data type (e.g., integers, strings, floats). It can be created from Python data structures like lists, dictionaries, or a list of dictionaries.
Example: Create DataFrame using dictionary:
Python
import pandas as pd
data = {'Name': ['Alice', 'Bob', 'Charlie'],'Age': [25, 30, 35]}
df = pd.DataFrame(data)
print(df)
Output Name Age
0 Alice 25
1 Bob 30
2 Charlie 35
In this example, a dictionary named data is created with keys representing column names (Name, Age) and values as lists containing the respective data. pd.DataFrame() function is then used to convert this dictionary into a DataFrame, which is stored in the variable df.
To learn Different ways to create dataframe, refer: Creating a Pandas DataFrame page.
- Series in Pandas is 1-dimensional labeled array capable of holding any data type (integers, strings, floats, etc.). Each element is associated with an index, either default (0, 1, 2…) or custom labels. It can be created from lists, NumPy arrays, dictionaries, or scalar values.
Note: All elements must be of the same data type.
Example: Creating a Series
Python
import pandas as pd
age_series = pd.Series([25, 30, 35], index=['Alice', 'Bob', 'Charlie'])
print(age_series)
OutputAlice 25
Bob 30
Charlie 35
dtype: int64
To Learn more ways to creating pandas series, refer: Creating a Pandas Series.
Section 3 : Reading CSV Files
CSV (Comma Separated Values) files are a common format for storing large datasets in plain text. The Pandas library in Python provides, read_csv() function, to load these files into a DataFrame. For our example we will use people.csv.
Python
import pandas as pd
df = pd.read_csv("people.csv")
print(df)
Output:

Pandas Read CSV
Note: With this line, pandas will display only the first and last 5 rows if the DataFrame is large.
Pandas provides flexible options to work with various file formats, including CSV and JSON (JavaScript Object Notation). Using read_json() function in pandas, we can read JSON files. To load a JSON file into a Pandas DataFrame, use the following code:
Python
import pandas as pd
# Load the JSON file into a DataFrame
df = pd.read_json('data.json')
print(df)
Note: To run the code and learn about how the read_json() function handles different JSON structures with various parameters, refer to the page: Reading JSON Files.
Section 4: Understanding and Analyzing the Data Frame
After creating or loading a DataFrame, inspecting and summarizing the data is an important step in understanding dataset. Pandas provides various functions to help you view and analyze the data.
head(): View the first n rows of the DataFrame (default is 5 rows).tail(): View the last n rows of the DataFrame (default is 5 rows).info(): This method provides a concise summary of the DataFrame, including the number of non-null entries, column names, and data types. Let’s see an example demonstrating the use of .head(), .tail(), and .info() methods:
Python
import pandas as pd
data = {'Name': ['Alice', 'Bob', 'Charlie', 'David', 'Eva'],
'Age': [24, 27, 22, 32, 29],
'City': ['New York', 'Los Angeles', 'Chicago', 'Houston', 'Phoenix']}
df = pd.DataFrame(data)
# View the first 3 rows of the DataFrame
print("First 3 rows using head():")
print(df.head(3))
# View the last 2 rows of the DataFrame
print("\nLast 2 rows using tail():")
print(df.tail(2))
# Get a concise summary of the DataFrame
print("\nDataFrame summary using info():")
df.info()
Output:

Viewing and Inspecting Data With Pandas
Pandas offers versatile tools for viewing and inspecting data, enabling to quickly summarize or dive deeper into specific data attributes. To explore more advanced functionalities in Pandas, you can follow the links below:
Note: describe generates descriptive statistics for numerical columns (such as count, mean, standard deviation, etc.). You can apply it to the entire DataFrame or specific columns. (You can also use .describe(include='all') to include non-numeric data in the summary.)
Section 5: Indexing in Pandas
Indexing in Pandas refers to process of accessing and selecting data from a Pandas DataFrame or Series. There are multiple ways to do this. We will cover how to to basic indexing, select specific columns , apply slicing, and use Boolean indexing to filter data efficiently.
Example : Basic Indexing ( Selecting a single column ) with use of [ ] operator:
Python
import pandas as pd
data = {'Name': ['Alice', 'Bob', 'Charlie'],'Age': [25, 30, 35]}
df = pd.DataFrame(data)
# Select a single column
age_column = df['Age']
print(age_column)
We can also select multiple columns by passing a list of column names. For example, you can select both the ‘Name’ and ‘Age’ columns by providing a list [‘Name’, ‘Age’]. When it comes to selecting rows: we can use .loc[] to select rows by label, meaning we refer to the row index or label directly. Alternatively, .iloc[] allows for position-based indexing, where we select rows by their integer positions.
Python
import pandas as pd
data = {'Name': ['Alice', 'Bob', 'Charlie'], 'Age': [25, 30, 35]}
df = pd.DataFrame(data)
# Using .loc[] to select rows by label
row_by_label = df.loc[1] # Selects the row with index label 1 (Bob's data)
# Using .iloc[] to select rows by position
row_by_position = df.iloc[1] # Selects the second row (Bob's data)
print("Row by label:\n", row_by_label)
print("Row by position:\n", row_by_position)
For more examples where we have covered selecting single and multiple columns and row based selection: Indexing and Selecting Data with Pandas. Explore the following resources for advanced indexing techniques:
Section 6: Selecting and Filtering Data – Techniques to Filter Based on Conditions
For selecting and filtering the data after indexing: is about narrowing down the data by applying conditions to select only those rows or columns that meet certain criteria.
Python
import pandas as pd
data = {'Name': ['Alice', 'Bob', 'Charlie'],'Age': [25, 30, 35]}
df = pd.DataFrame(data)
# Filtering rows where Age is greater than 30
filtered_df = df[df['Age'] > 28]
print(filtered_df)
In this example, we filter the rows where the Age column is greater than 30. This is a simple condition that helps narrow down the data to only those rows that meet the condition. For more methods, refer to our page : Selecting Rows from a Dataframe based on Column Values. You can also refer to following resources for advanced techniques and more conditions in filtering and selection:
Section 7: Dealing with Rows and Columns in Pandas DataFrame
Involves various data manipulation techniques in Pandas including adding and deleting columns, truncating data, iterating over DataFrames, and sorting data. Dealing with Rows and Columns in Pandas DataFrame – every concept and step is explained in this post.
1. Adding a New Column to DataFrame: There are several methods available, each suitable for specific use cases. We can easily add new columns by assigning values to them by direct assignment.
Python
import pandas as pd
df = pd.DataFrame({'Name': ['Alice', 'Bob', 'Charlie'], 'Age': [25, 30, 35]})
# Adding a new column with a list of values
df['Salary'] = [50000, 60000, 70000]
print(df)
Output Name Age Salary
0 Alice 25 50000
1 Bob 30 60000
2 Charlie 35 70000
There are multiple methods, for that refer to: Adding new column to existing dataFrame in pandas. If you want to add columns from one dataFrame to another, refer to Adding Columns from Another DataFrame.
2. Renaming columns using the rename() method or by directly modifying the columns attribute.
- Use
rename() for selective renaming. - Use
columns attribute for renaming all columns at once.
Python
import pandas as pd
df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6], 'C': [7, 8, 9]})
# Renaming columns
df.rename(columns={'A': 'X', 'B': 'Y', 'C': 'Z'}, inplace=True)
print(df)
Output X Y Z
0 1 4 7
1 2 5 8
2 3 6 9
We can also: Rename column by index in Pandas, refer to the article for examples.
3. Reindexing Data with Pandas : allows to change the row and column labels of a DataFrame or Series. This operation is useful for aligning data with a new set of labels, handling missing data, and reorganizing datasets. The reindex() method can change the row indices or column labels of a DataFrame.
Note: If the new index includes labels not present in the original DataFrame, the corresponding values will be set to NaN by default.
Python
import pandas as pd
# Sample DataFrame
data = {'A': [10, 20, 30], 'B': [40, 50, 60]}
df = pd.DataFrame(data)
# Reindex rows
new_index = [0, 1, 2, 3]
df_reindexed = df.reindex(new_index)
print("Reindexed Rows:\n", df_reindexed)
OutputReindexed Rows:
A B
0 10.0 40.0
1 20.0 50.0
2 30.0 60.0
3 NaN NaN
What if you want to reset the index? for that refer to: Convert Index to Column in Pandas Dataframe
Section 8: Handling Missing Data in Pandas
Working with Missing Data in Pandas is one of the most common tasks in data manipulation. Pandas provides various functions to identify, fill, and drop missing values efficiently. To explore the procedure hand-to-hand in detail refer to below steps:
1. Identifying Missing Data With Pandas using:
isnull(): Returns True where values are missing (NaN), and False where values are present.notnull(): Returns True where ues are present, and False where they are missing.
Here’s an example:
Python
import pandas as pd
import numpy as np
df = pd.DataFrame({'Col1': [1, 2, np.nan],'Col2': [3, np.nan, np.nan]})
# Check for missing values
print(df.isnull())
# Output:
# Col1 Col2
# 0 False False
# 1 False True
# 2 True True
2. Filling Missing Data: For Replace missing values with a specific value, fillna() is used. For example: you want to fill missing values with default value 0 you can perform it like : df.fillna(0).
Python
import pandas as pd
import numpy as np
df = pd.DataFrame({'Col1': [1, 2, np.nan],'Col2': [3, np.nan, np.nan]})
df.fillna(0)
print(df)
We can also fill missing values with below techniques:
3. Dropping Missing Data With Pandas: We can also remove rows or columns that contain missing values using .dropna(). For example:
Python
import pandas as pd
data = {'Name': ['Alice', 'Bob', None], 'Age': [25, None, 35]}
df = pd.DataFrame(data)
# Drop rows with missing values
df_dropped = df.dropna()
print(df_dropped)
We can also use df.dropna(axis=1) to remove columns or df.dropna(subset=[‘Column1’, ‘Column2’]) to drop rows with missing values in specific columns. For understand various ways for which we can use drop refer to:
Section 9: Aggregation and Grouping With Pandas
Aggregation and grouping in Pandas are powerful tools for analyzing and summarizing data. Grouping allows to segment your data into categories, while aggregation performs operations (like sum, mean, or count) on these groups to derive insights. groupby() function is commonly used for grouping data, followed by aggregation methods like sum(), mean(), or custom functions for statistical analysis.
Examples of operations:
Python
import pandas as pd
data = {'Category': ['A', 'B', 'A', 'B'], 'Value': [10, 20, 30, 40]}
df = pd.DataFrame(data)
# Group data by 'Category' and calculate the sum
grouped_sum = df.groupby('Category').sum()
print("Sum:\n", grouped_sum)
# Compute the mean for each group
grouped_mean = df.groupby('Category')['Value'].mean()
print("\nMean:\n", grouped_mean)
# Aggregate using multiple functions
grouped_agg = df.groupby('Category').agg(['sum', 'mean'])
print("\nAggregated:\n", grouped_agg)
In the example above:
groupby('Category'): Groups data by the ‘Category’ column..sum(), .mean(), and .size(): Perform aggregation on the grouped data.
Section 10: Merging and Concatenating Data in Pandas
This section covers techniques for combining multiple DataFrames or Series into a single DataFrame. These operations are essential for integrating datasets and can be performed in several ways:
1. Merging DataFrames: Combines data based on common column or index using functions like merge or join. There are 4 types of joins:
- Inner Join: Keep rows that match in both DataFrames.
- Left Join: Keep all rows from the left DataFrame and match data from the right.
- Right Join and Outer Join are similar but differ in data retention rules.
Refer : Different types of Joins in pandas
Example of operations:
Python
import pandas as pd
data = {'Name': ['Alice', 'Charlie', 'Edward', 'Grace'],
'Years_Experience': [2, 3, 4, 6],
'Role': ['Manager', 'Analyst', 'Developer', 'HR']}
df = pd.DataFrame(data)
# Merging DataFrames on the 'Name' column (inner join)
df_merged_inner = pd.merge(df1, df3, on='Name', how='inner')
# Merging DataFrames with outer join
df_merged_outer = pd.merge(df1, df3, on='Name', how='outer')
print("\nInner Merge (Intersection of Names):\n", df_merged_inner)
print("\nOuter Merge (Union of Names):\n", df_merged_outer)
For examples of all operations:
2. Concatenating Data: Refers to stacking DataFrames either vertically (adding rows) or horizontally (adding columns). This can be achieved using the pd.concat() function. Let’s create two dataFrames and concatenate it with the original one:
Python
import pandas as pd
data = {'Name': ['Alice', 'Charlie', 'Edward', 'Grace'],'Years_Experience': [2, 3, 4, 6],'Role': ['Manager', 'Analyst', 'Developer', 'HR']}
df = pd.DataFrame(data)
# New DataFrame to concatenate
new_data = {'Name': ['John', 'Lily'],'Years_Experience': [5, 3],'Role': ['Designer', 'Developer']}
new_df = pd.DataFrame(new_data)
# Concatenate the original and new DataFrames along rows (axis=0)
concatenated_df = pd.concat([df, new_df], axis=0, ignore_index=True)
print("Concatenated DataFrame:\n", concatenated_df)
In the code:
- The
axis=0 parameter specifies concatenation along rows (vertically). ignore_index=True resets the index so it doesn’t retain the original indices from both DataFrames.
Learn Concatenate Two or More Pandas DataFrames with all operations and multiple examples. For more, Go Through below articles:
Section 11: Reshaping Data in Pandas
Reshaping data involves changing the structure of rows and columns to better organize or analyze data. Common operations include:
1. Pivot Tables in Pandas: Reshape data based on column values. The .pivot_table() method is particularly powerful for creating aggregated views of data.
Python
import pandas as pd
data = {'Date': ['2024-10-01', '2024-10-01', '2024-10-02', '2024-10-02'],'Category': ['A', 'B', 'A', 'B'],'Values': [10, 20, 15, 25]}
df = pd.DataFrame(data)
pivot_table = df.pivot_table(values='Values', index='Date', columns='Category', aggfunc='sum')
print(pivot_table)
For more examples and in-depth understanding: Pivoting tables in pandas
2. Melting and Unmelting: Multiple columns are combined into a single key-value pair.
When you melt data:
- The column names become the keys.
- The values in the columns become the values.
Note: To melt data, you must specify columns that act as “identifiers” (id_vars) and others that need to be melted (value_vars).
Python
import pandas as pd
data = {'Date': ['2024-10-01', '2024-10-01', '2024-10-02', '2024-10-02'],'Category': [10, None, 15, None],'Category_B': [None, 20, None, 25]}
df = pd.DataFrame(data)
# Melt data from wide to long format
melt_df = df.melt(id_vars='Date', var_name='Category', value_name='Values')
print(melt_df)
More examples with : Reshaping – Melt and Unmelt
3. Stacking and Unstacking With Pandas: Reshape the data by changing its rows and columns, particularly when working with MultiIndex DataFrames. These techniques help pivot the layout of your data.
- Stacking: Moves the data from columns to rows (making the DataFrame taller).
- Unstacking: Moves the data from rows to columns (making the DataFrame wider). Let’s take a simple DataFrame with multiple levels of columns.
Python
import pandas as pd
data = {
('Sales', 'Q1'): [100, 150],
('Sales', 'Q2'): [200, 250],
('Expenses', 'Q1'): [50, 70],
('Expenses', 'Q2'): [80, 90]
}
index = ['Product_A', 'Product_B']
df = pd.DataFrame(data, index=index)
print(df)
stacked_df = df.stack() # Columns → Rows
print(stacked_df)
unstacked_df = stacked_df.unstack() # Rows → Columns
print(unstacked_df)
Explanation:
- Columns are stacked into rows.
- Now, the data has a MultiIndex for rows:
- First level: Product (e.g., Product_A, Product_B)
- Second level: Category (e.g., Sales, Expenses).
- Q1 and Q2 remain as columns
Let’s compare and understand in-depth with more examples : stack,unstack and melt method. We can also use transpose method for Swapping Rows and Columns
Section 12: Handling Large Datasets With Pandas
When working with large datasets or performing intensive computations, optimizing performance in Pandas is required. Below are some techniques to improve the efficiency of your data processing workflows.
- Working with Chunks of Data: When dealing with large datasets that may not fit into memory, you can use the
chunksize parameter in pd.read_csv() to load the data in chunks that allows to process the data in manageable pieces rather than loading everything at once. - Optimizing Memory Usage by Changing Data Types: By default, Pandas may use larger data types than necessary (e.g.,
int64 instead of int8), which can lead to inefficient memory usage. Changing the data types with .astype() can optimize memory consumption significantly, especially with large datasets.
For more methods, click on link.
Refer to Pandas Exercises and Programs for hands-on practice to reinforce your understanding of key concepts, including data manipulation, cleaning, and analysis.
Python Pandas Tutorial – FAQs
What is pandas used for in python?
Python Pandas is used for data manipulation, analysis, and cleaning. It simplifies handling structured data like spreadsheets and SQL tables. With Pandas, you can import data, clean it, transform it, and perform operations such as grouping, merging, and aggregating.
Is Python pandas easy to learn?
Yes, Python Pandas is relatively easy to learn, especially for those with basic Python knowledge.
How to start with Python pandas?
To start with Pandas, you can follow an introduction to Pandas Python guide.
Is pandas harder than SQL?
Pandas and SQL serve different purposes, but neither is inherently harder. SQL is used for database queries, while Pandas provides more flexibility for in-memory data manipulation. If you’re familiar with SQL queries, you might find Pandas query examples helpful to bridge the gap between SQL and Pandas.
The full form of Pandas is “Python Data Analysis Library,” derived from the term “panel data.” It’s designed for data manipulation and analysis, making it indispensable in data science tasks
What are Pandas best used for?
Pandas are best used for tasks like data cleaning, transformation, and analysis. You can work with time-series data, perform merging, and handle missing values with ease.
When to use pandas?
Use Pandas when working with structured datasets like spreadsheets, databases, or CSV files. It’s ideal for data cleaning, transformation, analysis, and visualization.
Get IBM Certification and a 90% fee refund on completing 90% course in 90 days! Take the Three 90 Challenge today.
Master Data Analysis using Excel, SQL, Python & PowerBI with this complete program and also get a 90% refund. What more motivation do you need? Start the challenge right away!
Similar Reads
Pandas Tutorial
Pandas is a powerful data manipulation and analysis library for Python. It provides data structures like series and dataframes to effectively easily clean, transform, and analyze large datasets and integrates seamlessly with other python libraries, such as numPy and matplotlib. It offers powerful fu
15+ min read
Creating Objects
Creating a Pandas DataFrame
In the real world, a Pandas DataFrame will be created by loading the datasets from existing storage, storage can be SQL Database, CSV file, and Excel file. Pandas DataFrame can be created from the lists, dictionary, and from a list of dictionary etc. A Dataframe is a two-dimensional data structure,
3 min read
Python Pandas Series
Pandas Series is a one-dimensional labeled array capable of holding data of any type (integer, string, float, python objects, etc.). Pandas Series Examples[GFGTABS] Python # import pandas as pd import pandas as pd # simple array data = [1, 2, 3, 4] ser = pd.Series(data) print(ser) [/GFGTABS]Output0
10 min read
Creating a Pandas Series
Pandas Series is a one-dimensional labeled array capable of holding data of any type (integer, string, float, python objects, etc.). The axis labels are collectively called index. Labels need not be unique but must be a hashable type. The object supports both integer and label-based indexing and pro
4 min read
Selection & Slicing
Dealing with Rows and Columns in Pandas DataFrame
A Data frame is a two-dimensional data structure, i.e., data is aligned in a tabular fashion in rows and columns. We can perform basic operations on rows/columns like selecting, deleting, adding, and renaming. In this article, we are using nba.csv file. Dealing with Columns In order to deal with col
5 min read
Python | Pandas Extracting rows using .loc[]
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages and makes importing and analyzing data much easier. Pandas provide a unique method to retrieve rows from a Data frame. DataFrame.loc[] met
3 min read
Extracting rows using Pandas .iloc[] in Python
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages that makes importing and analyzing data much easier. here we are learning how to Extract rows using Pandas .iloc[] in Python. Pandas .iloc
7 min read
Indexing and Selecting Data with Pandas
Indexing in Pandas refers to selecting specific rows and columns from a DataFrame. It allows you to subset data in various ways, such as selecting all rows with specific columns, some rows with all columns, or a subset of both rows and columns. This technique is also known as Subset Selection. Let's
6 min read
Boolean Indexing in Pandas
In boolean indexing, we will select subsets of data based on the actual values of the data in the DataFrame and not on their row/column labels or integer locations. In boolean indexing, we use a boolean vector to filter the data. Boolean indexing is a type of indexing that uses actual values of the
6 min read
Python | Pandas DataFrame.ix[ ]
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages and makes importing and analyzing data much easier. Pandas DataFrame.ix[ ] is both Label and Integer based slicing technique. Besides pure
2 min read
Python | Pandas Series.str.slice()
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric python packages. Pandas is one of those packages and makes importing and analyzing data much easier. Pandas str.slice() method is used to slice substrings from a string present in Pandas
3 min read
How to take column-slices of DataFrame in Pandas?
In this article, we will learn how to slice a DataFrame column-wise in Python. DataFrame is a two-dimensional tabular data structure with labeled axes. i.e. columns. Creating Dataframe to slice columns[GFGTABS] Python # importing pandas import pandas as pd # Using DataFrame() method from pandas modu
2 min read
Operations
Python | Pandas.apply()
Pandas.apply allow the users to pass a function and apply it on every single value of the Pandas series. It comes as a huge improvement for the pandas library as this function helps to segregate data according to the conditions required due to which it is efficiently used in data science and machine
4 min read
Apply function to every row in a Pandas DataFrame
Python is a great language for performing data analysis tasks. It provides a huge amount of Classes and functions which help in analyzing and manipulating data more easily. In this article, we will see how we can apply a function to every row in a Pandas Dataframe. Apply Function to Every Row in a P
7 min read
Python | Pandas Series.apply()
Pandas series is a One-dimensional ndarray with axis labels. The labels need not be unique but must be a hashable type. The object supports both integer- and label-based indexing and provides a host of methods for performing operations involving the index. Pandas Series.apply() function invoke the p
3 min read
Python | Pandas dataframe.aggregate()
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages and makes importing and analyzing data much easier. Dataframe.aggregate() function is used to apply some aggregation across one or more co
3 min read
Pandas DataFrame mean() Method
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages and makes importing and analyzing data much easier. Pandas DataFrame mean() Pandas dataframe.mean() function returns the mean of the value
2 min read
Python | Pandas Series.mean()
Pandas series is a One-dimensional ndarray with axis labels. The labels need not be unique but must be a hashable type. The object supports both integer- and label-based indexing and provides a host of methods for performing operations involving the index. Pandas Series.mean() function return the me
2 min read
Python | Pandas dataframe.mad()
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric python packages. Pandas is one of those packages and makes importing and analyzing data much easier. Pandas dataframe.mad() function return the mean absolute deviation of the values for t
2 min read
Python | Pandas Series.mad() to calculate Mean Absolute Deviation of a Series
Pandas provide a method to make Calculation of MAD (Mean Absolute Deviation) very easy. MAD is defined as average distance between each value and mean. The formula used to calculate MAD is: Syntax: Series.mad(axis=None, skipna=None, level=None) Parameters: axis: 0 or ‘index’ for row wise operation a
2 min read
Python | Pandas dataframe.sem()
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric python packages. Pandas is one of those packages and makes importing and analyzing data much easier. Pandas dataframe.sem() function return unbiased standard error of the mean over reques
3 min read
Python | Pandas Series.value_counts()
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric python packages. Pandas is one of those packages and makes importing and analyzing data much easier. Pandas series is a One-dimensional ndarray with axis labels. The labels need not be un
2 min read
Python | Pandas Index.value_counts()
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric python packages. Pandas is one of those packages and makes importing and analyzing data much easier.Pandas Index.value_counts() function returns object containing counts of unique values.
2 min read
Applying Lambda functions to Pandas Dataframe
In Python Pandas, we have the freedom to add different functions whenever needed like lambda function, sort function, etc. We can apply a lambda function to both the columns and rows of the Pandas data frame. Syntax: lambda arguments: expression An anonymous function which we can pass in instantly w
6 min read
Manipulating Data
Adding New Column to Existing DataFrame in Pandas
Adding a new column to a DataFrame in Pandas is a simple and common operation when working with data in Python. You can quickly create new columns by directly assigning values to them. Let's discuss how to add new columns to the existing DataFrame in Pandas. There can be multiple methods, based on d
6 min read
Python | Delete rows/columns from DataFrame using Pandas.drop()
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages which makes importing and analyzing data much easier. In this article, we will how to delete a row in Excel using Pandas as well as delete
4 min read
Python | Pandas DataFrame.truncate
Pandas DataFrame is a two-dimensional size-mutable, potentially heterogeneous tabular data structure with labeled axes (rows and columns). Arithmetic operations align on both row and column labels. It can be thought of as a dict-like container for Series objects. This is the primary data structure o
3 min read
Python | Pandas Series.truncate()
Pandas series is a One-dimensional ndarray with axis labels. The labels need not be unique but must be a hashable type. The object supports both integer- and label-based indexing and provides a host of methods for performing operations involving the index. Pandas Series.truncate() function is used t
2 min read
Iterating over rows and columns in Pandas DataFrame
Iteration is a general term for taking each item of something, one after another. Pandas DataFrame consists of rows and columns so, to iterate over dataframe, we have to iterate a dataframe like a dictionary. In a dictionary, we iterate over the keys of the object in the same way we have to iterate
7 min read
Pandas Dataframe.sort_values()
In Pandas, sort_values() function sorts a DataFrame by one or more columns in ascending or descending order. This method is essential for organizing and analyzing large datasets effectively. Syntax: DataFrame.sort_values(by, axis=0, ascending=True, inplace=False, kind='quicksort', na_position='last'
2 min read
Python | Pandas Dataframe.sort_values() | Set-2
Prerequisite: Pandas DataFrame.sort_values() | Set-1 Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages, and makes importing and analyzing data much easier. Pandas sort_values() function so
3 min read
How to add one row in existing Pandas DataFrame?
Adding rows to a Pandas DataFrame is a common task in data manipulation and can be achieved using methods like loc[], and concat(). Method 1. Using loc[] - By Specifying its Index and ValuesThe loc[] method is ideal for directly modifying an existing DataFrame, making it more memory-efficient compar
4 min read
Merging, Joining, Concatenating and Comparing
Python | Pandas Merging, Joining, and Concatenating
Pandas DataFrame is a two-dimensional size-mutable, potentially heterogeneous tabular data structure with labelled axes (rows and columns). A Data frame is a two-dimensional data structure, i.e., data is aligned in a tabular fashion in rows and columns. We can join, merge, and concat dataframe using
11 min read
Python | Pandas Series.str.cat() to concatenate string
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric python packages. Pandas is one of those packages and makes importing and analyzing data much easier.Pandas str.cat() is used to concatenate strings to the passed caller series of string.
3 min read
Python - Pandas dataframe.append()
Pandas append function is used to add rows of other dataframes to end of existing dataframe, returning a new dataframe object. Columns not in the original data frames are added as new columns and the new cells are populated with NaN value. Append Dataframe into another DataframeIn this example, we a
5 min read
Python | Pandas Series.append()
Pandas series is a One-dimensional ndarray with axis labels. The labels need not be unique but must be a hashable type. The object supports both integer- and label-based indexing and provides a host of methods for performing operations involving the index. Pandas Series.append() function is used to
4 min read
Python | Pandas Index.append()
Python is an excellent language for data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas are one of those packages, making importing and analyzing data much easier. Pandas Index.append() The function is used to append a single or a collection of indices
2 min read
Python | Pandas Series.combine()
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages and makes importing and analyzing data much easier. Pandas Series.combine() is a series mathematical operation method. This is used to com
3 min read
Add a row at top in pandas DataFrame
Pandas DataFrame is two-dimensional size-mutable, potentially heterogeneous tabular data structure with labeled axes (rows and columns). Let's see how can we can add a row at top in pandas DataFrame.Observe this dataset first. C/C++ Code # importing pandas module import pandas as pd # making data fr
1 min read
Python | Pandas str.join() to join string/list elements with passed delimiter
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages and makes importing and analyzing data much easier. Pandas str.join() method is used to join all elements in list present in a series with
2 min read
Join two text columns into a single column in Pandas
Let's see the different methods to join two text columns into a single column. Method #1: Using cat() function We can also use different separators during join. e.g. -, _, " " etc. # importing pandas import pandas as pd df = pd.DataFrame({'Last': ['Gaitonde', 'Singh', 'Mathur'], 'First': ['Ganesh',
1 min read
How To Compare Two Dataframes with Pandas compare?
A DataFrame is a 2D structure composed of rows and columns, and where data is stored into a tubular form. It is mutable in terms of size, and heterogeneous tabular data. Arithmetic operations can also be performed on both row and column labels. To know more about the creation of Pandas DataFrame. He
5 min read
How to compare the elements of the two Pandas Series?
Sometimes we need to compare pandas series to perform some comparative analysis. It is possible to compare two pandas Series with help of Relational operators, we can easily compare the corresponding elements of two series at a time. The result will be displayed in form of True or False. And we can
3 min read
Working with Date and Time
Python | Working with date and time using Pandas
While working with data, encountering time series data is very usual. Pandas is a very useful tool while working with time series data. Pandas provide a different set of tools using which we can perform all the necessary tasks on date-time data. Let's try to understand with the examples discussed b
8 min read
Python | Pandas Timestamp.timestamp
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages and makes importing and analyzing data much easier. Pandas Timestamp.timestamp() function returns the time expressed as the number of seco
3 min read
Python | Pandas Timestamp.now
Python is a great language for data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages that makes importing and analyzing data much easier. Pandas Timestamp.now() function returns the current time in the local timezone. It is Equiv
3 min read
Python | Pandas Timestamp.isoformat
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages and makes importing and analyzing data much easier. Pandas Timestamp objects represent date and time values, making them essential for wor
2 min read
Python | Pandas Timestamp.date
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric python packages. Pandas is one of those packages and makes importing and analyzing data much easier. Pandas Timestamp.date() function return a datetime object with same year, month and da
2 min read
Python | Pandas Timestamp.replace
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages that makes importing and analyzing data much easier. Pandas Timestamp.replace() function is used to replace the member values of the given
3 min read
Python | Pandas.to_datetime()
When a CSV file is imported and a Data Frame is made, the Date time objects in the file are read as a string object rather than a Date Time object Hence it’s very tough to perform operations like Time difference on a string rather than a Date Time object. Pandas to_datetime() method helps to convert
4 min read
Python | pandas.date_range() method
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages that makes importing and analyzing data much easier. pandas.date_range() is one of the general functions in Pandas which is used to return
4 min read
Working With Text Data
Python | Pandas Working With Text Data
Series and Indexes are equipped with a set of string processing methods that make it easy to operate on each element of the array. Perhaps most importantly, these methods exclude missing/NA values automatically. These are accessed via the str attribute and generally, have names matching the equivale
8 min read
Python | Pandas Series.str.lower(), upper() and title()
Python is a great language for data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages, making importing and analyzing data much easier. Python has some inbuilt methods to convert a string into a lower, upper, or Camel case. But th
4 min read
Python | Pandas Series.str.replace() to replace text in a series
Python is a great language for data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages that makes importing and analyzing data much easier. Pandas Series.str.replace() method works like Python .replace() method only, but it works o
5 min read
Python | Pandas Series.replace()
Pandas series is a One-dimensional ndarray with axis labels. The labels need not be unique but must be a hashable type. The object supports both integer- and label-based indexing and provides a host of methods for performing operations involving the index. Pandas Series.replace() function is used to
3 min read
Python | Pandas Series.str.strip(), lstrip() and rstrip()
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages that makes importing and analyzing data much easier. Pandas provide 3 methods to handle white spaces(including New lines) in any text data
4 min read
Python | Pandas tseries.offsets.DateOffset
Dateoffsets are a standard kind of date increment used for a date range in Pandas. It works exactly like relative delta in terms of the keyword args we pass in. DateOffsets work as follows, each offset specifies a set of dates that conform to the DateOffset. For example, Bday defines this set to be
4 min read
Working with CSV and Excel files
Applications and Projects
How to Do a vLookup in Python using pandas
Vlookup is essentially used for vertically arranged data. Vlookup is an operation used to merge 2 different data tables based on some condition where there must be at least 1 common attribute(column) between the two tables. After performing this operation we get a table consisting of all the data fr
3 min read
Convert CSV to HTML Table in Python
CSV file is a Comma Separated Value file that uses a comma to separate values. It is basically used for exchanging data between different applications. In this, individual rows are separated by a newline. Fields of data in each row are delimited with a comma.Example : Name, Salary, Age, No.of years
2 min read
KDE Plot Visualization with Pandas and Seaborn
Kernel Density Estimate (KDE) plot, a visualization technique that offers a detailed view of the probability density of continuous variables. In this article, we will be using Iris Dataset and KDE Plot to visualize the insights of the dataset. What is KDE Plot?KDE Plot described as Kernel Density Es
4 min read
Analyzing selling price of used cars using Python
Now-a-days, with the technological advancement, Techniques like Machine Learning, etc are being used on a large scale in many organisations. These models usually work with a set of predefined data-points available in the form of datasets. These datasets contain the past/previous information on a spe
7 min read
Add CSS to the Jupyter Notebook using Pandas
In Jupyter Notebook, when we print the output table of our data, it shows a very basic table containing the data. But what if we want to customize this default style? In this article, we will see how we can add styles to our output data table. This is how a default data table looks like in Jupyter N
2 min read
Pandas Interview Questions
Panda is a FOSS (Free and Open Source Software) Python library which provides high-performance data manipulation, in Python. It is used in various areas like data science and machine learning. Pandas is not just a library, it's an essential skill for professionals in various domains, including finan
15+ min read