Data Processing in Python

Introduction

Welcome to this book on data processing in Python! This course is designed to help you learn the basics of how to use Python for data analysis and data-driven decision making. In this book, we will cover the fundamentals of working with data in Python, such as loading, cleaning, exploring, and transforming it. We will also discuss the most popular libraries used for data processing tasks, such as NumPy and Pandas. Finally, we’ll be looking at several case studies that will allow us to explore the different options available when it comes to analysing and interpreting data using Python. By the end of this book, you should have a good understanding of how to use Python for any kind of data-related task. So, let us get started!

Objectives

1. Understand the fundamentals of data manipulation with Python.
2. Develop an understanding of data processing tools and techniques such as NumPy, Pandas, and Scikit-learn.
3. Learn to read, manipulate, query and visualize structured data using Python libraries like matplotlib, seaborn and plotly.
4. Be able to use various data science libraries in Python to effectively analyze large datasets for meaningful insights.
5. Gain proficiency in building predictive models for supervised and unsupervised machine learning tasks such as classification, regression and clustering.

Table of Contents

Module 1: Introduction to Python

– Overview of Python fundamentals, including basic data types and control structures

– Using the Python interpreter and IDLE IDE

– Writing simple scripts in Python

– understanding of data processing tools and techniques such as NumPy, Pandas, and Scikit-learn.

Module 2: Working with Data in Python

– Creating lists, tuples, and dictionaries for storing data

– Working with string data types for processing text data

– Performing numerical calculations using numeric types and built-in functions

– Accessing and reading external files such as CSV, JSON, etc.

– Applying functional programming concepts such as map, filter, reduce.

Module 3: Introducing Pandas Library 

– Overview of Pandas library for working with datasets

– Learn to read, manipulate, query, and visualize structured data using Python libraries like matplotlib, seaborn and plotly

– Creating Pandas Series and Data Frames from different sources of data

– Manipulating rows, columns, indices & values in Data Frames

 Module 4: Analysing and Visualizing Data using Pandas.

– Filtering, selecting & aggregating data using slicing operations

– Performing elementary statistical operations on datasets

– Generating graphical representations of related variables

Module 5: Handling Big Data Sets with Dask

– Exploring dask library for parallel computing on large datasets

– Setting up dask client & cluster to work on distributed systems

– Optimizing code to maximize performance when dealing with big datasets

Module 6: Exams