Answering from Your Notes · practice
Split the Notes into Chunks
To retrieve the right part of a note, a program first needs the notes in parts. This lesson splits each note into paragraphs and numbers them, so that every passage has a short name such as 03-lists#4: the fourth paragraph of the lists note.
Splitting text on a separator
text.split(separator) cuts a string at every occurrence of separator and returns the pieces between them as a list. The separators themselves are dropped:
line = "ids,03-lists,06-dictionaries"
print(line.split(","))
print("one\n\ntwo\n\n\n\nthree".split("\n\n"))
print("# Title\n\nA first paragraph.\n".split("\n\n"))
Look closely at the second line. Four line breaks in a row hold two separators, so there is an empty piece between them. And the third line keeps the line break at the end of the text inside its last piece. Both are why the steps below strip every piece and skip the empty ones. In Markdown, a blank line separates paragraphs, and a blank line is two line breaks in a row, "\n\n".
Lesson 3 uses split() with no argument, which splits on any run of spaces and line breaks.
What a chunk is
Each note in notes/ is Markdown: a title line that starts with #, then paragraphs separated by blank lines. A code block in a note has no blank line inside it, so it stays in one piece. A chunk is one paragraph, stored as a dictionary:
{"note_id": "03-lists", "number": 4, "text": "Because a list is changed in place, ..."}
The title is not a chunk. It names the topic but answers nothing, so as a source it would only take the place of a paragraph that does.
Build split_note
split_note(note_id, text) returns the chunks of one note:
Split
texton"\n\n".For each piece, strip the spaces and line breaks from both ends with
.strip().Skip a piece that is empty or that starts with
#. Strip first, then check, so a title that came after an extra line break is still recognized.Number the pieces you keep from 1. Use a counter that starts at 0 and goes up by 1 for each kept piece, so a skipped title does not use up a number.
Append
{"note_id": note_id, "number": number, "text": piece}to a list, and return the list.
A note that holds only its title gives an empty list.
Build note_chunks
note_chunks(notes) takes the dictionary from load_notes and returns the chunks of every note in one flat list: all the chunks of the first note, then all the chunks of the second, and so on, in the dictionary’s order. Loop over the note ids, call split_note for each, and append each chunk it returns to one list. Appending the list split_note returns as one item would give a list of lists instead. Do not change notes.
Run it
Run python retrieval.py. The demonstration splits every note, prints how many chunks there are, and prints the chunks of the lists note:
The notes split into 76 chunks. The chunks of 03-lists:
{'note_id': '03-lists', 'number': 1, 'text': 'A list is an ordered collection of values. You write one with square brackets, such as `scores = [90, 72, 85]`. A list can hold values of any type, including a mix of types and other lists.'}
{'note_id': '03-lists', 'number': 2, 'text': 'Each item has a position, called its index, starting at 0. `scores[0]` is the first item and `scores[-1]` is the last. Reading an index that does not exist raises `IndexError`. A slice such as `scores[0:2]` returns a new list with the items from position 0 up to, but not including, position 2.'}
{'note_id': '03-lists', 'number': 3, 'text': 'Lists are mutable, so you can change them after creating them. `scores.append(60)` adds an item to the end, `scores.insert(0, 100)` adds one at a position, `scores.remove(72)` removes the first matching value, and `scores[1] = 75` replaces an item in place. These methods change the list and return `None`, so `scores = scores.append(60)` loses the list.'}
{'note_id': '03-lists', 'number': 4, 'text': 'Because a list is changed in place, two names that refer to the same list both see every change. Use `scores.copy()` or `list(scores)` when you need an independent list. Adding two lists with `+` also builds a new list and leaves both originals unchanged.'}
{'note_id': '03-lists', 'number': 5, 'text': '`len(scores)` tells you how many items a list holds, and `value in scores` tells you whether a value is one of them.'}
Run stopped at the next function to build: words is built in Lesson 3.
The twelve notes make 76 chunks, between 4 and 8 per note. The fourth chunk of the lists note is the one that answers “How do I make an independent copy of a list?”. The next lesson gives the program a way to find it.
What does "one\n\n\n\ntwo".split("\n\n") return?
A note is a title, then three paragraphs. What numbers do its chunks get?
Task
In retrieval.py, replace the unfinished bodies of two functions.
split_note(note_id, text): splittexton"\n\n". Strip each piece, and skip pieces that are empty or start with#. Number the kept pieces from 1 with a counter, and return a list of chunk dictionaries{"note_id": note_id, "number": number, "text": piece}, in the order the pieces appear.note_chunks(notes): return one flat list with the chunks of every note in the dictionarynotes, note by note in the dictionary’s order. Usesplit_notefor each note, and do not changenotes.
Run python retrieval.py to see how many chunks the notes make, then check your work.