AIBook
So You Want to Be a Data Scientist
Python Primer for Data Science
A complete path from your first import to code other people depend on. Twenty-five short chapters covering the Python language itself, NumPy and pandas, loading and cleaning real data, visualisation, the statistics you need to make an honest claim, and the scikit-learn interface. Written for someone who has never done data science, and assumes no mathematics beyond secondary school.
Getting Started
- 1A Language Worth LearningWhere Python came from, how a Christmas project named after a comedy troupe ended up running the world's science, and how the author got here by the slow route so that you do not have to.
- 2Should You Still Learn to Code?You are learning to program at the exact moment machines got good at programming. The honest answer concedes the premise completely, then asks what was ever the hard part.
- 3Why Python WonPython is a slow language that dominates a field obsessed with speed. Understanding that contradiction explains almost everything about how the tools are shaped and how you are meant to use them.
The Language
- 4Values, Types and NamesNumbers, text, true and false, and nothing. What a variable really is in Python, why some values can be changed and others cannot, and the one behaviour that surprises every beginner exactly once.
- 5Collections: Lists, Dicts, Sets and TuplesFour ways to hold several things at once. What each is good at, what each is terrible at, and how to pick between them by what the operation costs rather than by whichever one you learned first.
- 6Control Flow, Comprehensions and UnpackingLoops and conditions, then the comprehension syntax that replaces most of them. Plus the unpacking tricks that appear everywhere in data code and confuse everyone the first time they see them.
- 7Functions That BehaveArguments, defaults, scope and return values. Includes the mutable default trap, which catches every Python programmer exactly once, and the docstring habit that makes code survivable six months later.
- 8Iterators, Generators and Context ManagersHow Python processes things one at a time without loading everything at once, and the with statement that guarantees cleanup. This is the chapter that lets you read a file bigger than your memory.
- 9Errors, Modules and the Standard LibraryHow to fail usefully, how to split code across files without circular imports, and the seven standard library modules that come up constantly in data work.
The Numerical Stack
- 10NumPy: Thinking in ArraysWhat an array actually is in memory, why shape matters more than value, the difference between a view and a copy, and the broadcasting rules that make mismatched arrays work exactly as you intended.
- 11Pandas: The Eighty Per CentIn the real world you are never handed a clean dataset. What arrives has eleven spellings of one category, dates in three formats, and a merge that silently triples your row count. This is the chapter about that.
Working With Real Data
- 12Getting Data In: Files and FormatsLoading a file is the first thing you do and the first thing that goes wrong. Encodings, delimiters, the read_csv arguments that are not optional, and why Parquet should be your default for anything you open twice.
- 13Getting Data In: Databases and APIsWhen the data is not in a file. Connecting to a database without pulling the whole table, writing queries that cannot be injected into, and calling an API politely enough that it keeps answering.
- 14Cleaning and ReshapingThe unglamorous majority of the job. Finding what is wrong, deciding what missing means, dealing with duplicates and outliers, and turning wide data into long and back again.
- 15Joining Without Losing RowsJoins are where the most expensive mistakes happen, because they fail by producing a plausible result with the wrong number of rows and no error at all. Three lines prevent nearly all of it.
- 16Working With TextThe .str accessor, normalising the five spellings of one category, and enough regular expressions to be useful without needing a decoder ring next month.
- 17Dates, Times and Time ZonesParsing dates that arrive in three formats, storing them so they survive a daylight saving change, and the resampling that turns raw timestamps into something you can plot.
- 18Categories and EncodingThe categorical dtype that saves memory and prevents mistakes, and the four ways to turn a category into a number. One of those four leaks the target into your training data, which is why it needs its own section.
Seeing the Data
Making Claims
- 20The Statistics You Actually NeedNot a statistics course. The specific ideas that stop you believing a result that is not there, taught with simulation rather than proof, because seeing it happen a thousand times is more convincing than a formula.
- 21Evaluation, and How It Lies to YouThe most expensive mistake in this field is not building a bad model. It is building a bad model your own evaluation says is excellent, and finding out from a customer. This is the chapter to read twice.
Modelling
- 22scikit-learn: The Interface, Not the AlgorithmsOne API covers every model in the library, and learning that API is a Python skill rather than a machine learning one. This chapter teaches the contract, the Pipeline that makes leakage structurally impossible, and nothing at all about how a random forest works.
- 23Feature Engineering That SurvivesMost model improvement comes from better features, not better algorithms. The techniques that generalise, and the specific ways a feature that works in a notebook turns out to be impossible to compute in production.
Shipping
- 24From Notebook to Something That RunsA notebook is a research artifact. Getting to code that runs on a schedule, can be tested, and can be rolled back is a different discipline, and it is where most data science projects stall.
- 25Performance When It Actually MattersMost slow code is slow for one of about five reasons. How to find which one, the fixes in order of effort, and an honest account of when to stop optimising pandas and use something else.