6. File Handling

Learning Objectives

• Explain what file handling is and why programs need it to persist data beyond a single run
• Open, read, write, and append to files in Python using the correct mode strings
• Use the with statement to guarantee a file is closed even if an error occurs
• Distinguish text files from binary files and choose the right mode for each
• Check for file existence and delete files safely using the os module
• Identify and fix the most common file-handling bugs: forgetting to close a file, using the wrong mode, and not handling missing files

Quick Answer

File handling is how a program reads data from, and writes data to, files stored on disk instead of keeping everything in memory. Without it, every value your program computes would disappear the moment the program exits. File handling gives programs persistence — a game can save your progress, a text editor can save your document, and a web server can log every request it receives. In most languages you open a file to get a handle to it, perform reads or writes through that handle, and then close it to release the operating system resource. Python's with statement automates the closing step, which is why it's the standard way to work with files in modern Python code.

Why Programs Need Files

A running program's variables live in memory (RAM), and RAM is wiped clean the instant the program ends. If you want data to survive — a user's saved settings, a database of student records, a log of errors — you need to write it somewhere that outlives the process. A file on disk is the simplest form of that "somewhere."

Why It Matters

• Persistence — data survives after the program stops running.
• Sharing — files let different programs (or the same program run twice) exchange data without a network connection.
• Scale — files can hold far more data than reasonably fits in memory at once, and can be read piece by piece.
• Auditability — log files record what a system did, which is essential for debugging production issues.

A common misunderstanding: students assume a variable "saves" data just because it holds a value. It doesn't — the value only exists while the program is running. Only writing it to a file (or a database, which is built on the same idea) makes it durable.

Opening and Closing Files in Python

Python's open() function returns a file object that you use to read or write.

file_object = open(file_name, mode)

• file_name — the path to the file, relative or absolute.
• mode — a string describing what you intend to do with the file.

Mode	Meaning	If file doesn't exist	If file exists
'r'	Read (default)	Raises FileNotFoundError	Reads from the start
'w'	Write	Creates a new file	Overwrites all existing content
'a'	Append	Creates a new file	Adds new data after existing content
'rb' / 'wb'	Binary read/write	Same as above, but bytes instead of text	Same as above

file = open("scores.txt", "r")
content = file.read()
print(content)
file.close()  # must close manually when using open() directly

If you forget file.close(), the operating system may keep the file locked or lose buffered writes that haven't actually been flushed to disk yet — a real bug, not just untidy code.

The with Statement: The Safer Way

Python's with statement opens the file, hands it to you, and guarantees it gets closed when the block ends — even if an exception is raised inside the block.

with open("scores.txt", "r") as file:
    content = file.read()
    print(content)
# file is already closed here, automatically

This is why professional Python code almost always uses with open(...) instead of manually calling open() and close().

Reading a File Three Ways

with open("scores.txt", "r") as file:
    whole_thing = file.read()        # entire file as one string

with open("scores.txt", "r") as file:
    one_line = file.readline()       # just the next line

with open("scores.txt", "r") as file:
    all_lines = file.readlines()     # list of lines, e.g. ["Alice\n", "Bob\n"]
    for line in all_lines:
        print(line.strip())          # strip() removes the trailing newline

readlines() loads the whole file into a list, which is convenient but wasteful for huge files. For very large files, iterate directly over the file object — it reads one line at a time without loading everything into memory:

with open("huge_log.txt", "r") as file:
    for line in file:
        if "ERROR" in line:
            print(line.strip())

Writing and Appending

with open("scores.txt", "w") as file:
    file.write("Alice: 92\n")
    file.write("Bob: 85\n")
# scores.txt now contains exactly these two lines — anything that was there before is gone

with open("scores.txt", "a") as file:
    file.write("Carol: 78\n")
# Carol's line is added after Alice's and Bob's, nothing is erased

Real-world example: a simple attendance logger appends one line per class session instead of overwriting the file, so the historical record accumulates over the semester:

from datetime import date

def log_attendance(student_name):
    with open("attendance.log", "a") as file:
        file.write(f"{date.today()},{student_name}\n")

log_attendance("Priya")
log_attendance("Diego")

Checking Existence and Deleting Files

The os module handles filesystem-level operations that aren't about reading/writing content.

import os

if os.path.exists("scores.txt"):
    print("File exists")
    os.remove("scores.txt")
    print("File deleted")
else:
    print("File does not exist — nothing to delete")

Always check existence (or catch the exception) before deleting — calling os.remove() on a missing file raises FileNotFoundError and crashes the program if unhandled.

Text Files vs. Binary Files

# Text mode: Python decodes bytes into a string using an encoding (default UTF-8)
with open("notes.txt", "r") as file:
    text = file.read()          # returns a str

# Binary mode: no decoding happens — you get raw bytes
with open("photo.jpg", "rb") as file:
    data = file.read()          # returns a bytes object

Opening an image, audio file, or executable in text mode ('r') will either throw a UnicodeDecodeError or silently corrupt the data, because text mode tries to interpret arbitrary bytes as characters. Binary mode ('rb'/'wb') passes bytes through untouched.

File Handling in Other Languages

The concept is the same everywhere: open, operate, close.

#include <stdio.h>

int main() {
    FILE *file = fopen("scores.txt", "r");
    if (file == NULL) {
        printf("Could not open file\n");
        return 1;
    }
    char line[100];
    while (fgets(line, sizeof(line), file)) {
        printf("%s", line);
    }
    fclose(file);
    return 0;
}

import java.io.*;

public class ReadScores {
    public static void main(String[] args) throws IOException {
        try (BufferedReader reader = new BufferedReader(new FileReader("scores.txt"))) {
            String line;
            while ((line = reader.readLine()) != null) {
                System.out.println(line);
            }
        }
        // try-with-resources closes the file automatically, like Python's `with`
    }
}

Notice that C requires you to check for NULL manually because it has no exceptions, while Java's try-with-resources and Python's with both close the file automatically — a pattern called RAII-style resource management that shows up whenever a language deals with anything that must be released (files, network sockets, database connections).

Key Terms

Term	Definition	Related Concept
File Object	The handle returned by open() used to read or write a file	open(), File Mode
File Mode	A string ('r', 'w', 'a', 'rb', etc.) that tells open() how the file will be used	File Object
with Statement	A Python construct that automatically closes a resource when its block ends	Context Manager
Text File	A file whose content is interpreted as encoded characters (e.g., UTF-8)	Binary File
Binary File	A file whose content is read/written as raw bytes with no character decoding	Text File
Persistence	Data that survives after the program that created it has stopped running	File Handling
Buffering	Temporarily holding written data in memory before it is physically saved to disk	close(), Flushing
os.path.exists()	A function that checks whether a given file or directory path exists	os.remove()

Common Mistakes

Misconception: Calling open() is enough — the data is saved as soon as file.write() runs. Why it's wrong: Writes are often buffered in memory for performance and are only guaranteed to reach disk when the file is closed (or explicitly flushed). If the program crashes before close() runs, buffered data can be lost. Correct understanding: Always close the file (or use with, which closes it automatically) to guarantee that written data is actually flushed to disk.

---

Misconception: Opening a file in 'w' mode is safe to use whenever you want to "save more data" to a file. Why it's wrong: 'w' mode immediately erases all existing content the moment the file is opened, even if you never call write(). Using it when you meant 'a' silently destroys prior data with no warning. Correct understanding: Use 'a' (append) to add to existing content and 'w' (write) only when you intend to replace the file entirely.

---

Misconception: You can safely skip checking whether a file exists before reading it, since Python will "just handle it." Why it's wrong: Attempting to read a nonexistent file with 'r' mode raises an unhandled FileNotFoundError that crashes the program if there's no surrounding error handling. Correct understanding: Check os.path.exists() first, or wrap the file operation in a try/except FileNotFoundError block, so the program can respond gracefully instead of crashing.

Comparison and Connections

Feature	Text Mode ('r'/'w'/'a')	Binary Mode ('rb'/'wb')
Data type returned	str	bytes
Use case	.txt, .csv, .log, source code	Images, audio, executables, serialized data
Encoding involved	Yes (default UTF-8)	No — raw bytes pass through
Common error if misused	UnicodeDecodeError on binary data	Garbled/unreadable output on text data

Approach	Manual open()/close()	with Statement
Closes file on success	Yes, if you remember to call close()	Always, automatically
Closes file on exception	No — an unhandled exception skips close()	Yes — guaranteed even if an error occurs
Recommended for production code	No	Yes

Practice Questions

Recall

1. What is the difference between 'w' mode and 'a' mode when opening a file? Look for: 'w' overwrites/erases existing content (or creates a new file); 'a' adds new content to the end without touching what's already there.

2. Name the three methods used to read from a file object in Python, and what each returns. Look for: read() returns the whole file as one string; readline() returns the next single line; readlines() returns a list of all lines.

Understanding

3. Explain why using the with statement is preferred over manually calling open() and close(). Look for: with guarantees the file is closed even if an exception occurs inside the block, preventing resource leaks and lost buffered writes; manual close() can be skipped if an error happens first.

4. Why does opening a binary file (like a .jpg) in text mode cause problems? Look for: text mode tries to decode the raw bytes as characters using an encoding like UTF-8; binary data often doesn't form valid character sequences, causing a UnicodeDecodeError or corrupted data.

Application

5. Write a Python snippet that appends a new high score to "highscores.txt" without erasing previous scores. Look for: with open("highscores.txt", "a") as file: file.write(...) — using 'a' mode, not 'w'.

6. Write a snippet that safely deletes "temp.txt" only if it exists, without crashing if it doesn't. Look for: if os.path.exists("temp.txt"): os.remove("temp.txt"), or a try/except FileNotFoundError around os.remove().

Analysis

7. A student's program crashes with FileNotFoundError the first time it runs, because the log file hasn't been created yet. Diagnose the bug and propose a fix. Look for: the code opens the log file in 'r' mode, which requires the file to already exist; fix by opening in 'a' mode instead (which creates the file if missing) or by checking existence first and creating an empty file.

8. Compare reading a 10 GB log file with readlines() versus iterating over the file object directly (for line in file:). Which is better and why? Look for: readlines() loads the entire file into memory as a list, which can exhaust RAM for very large files; iterating directly over the file object reads one line at a time, using constant memory regardless of file size.

FAQ

Q: What happens if I open a file in 'w' mode but never call write()? The file is still truncated (emptied) the moment it's opened in 'w' mode, or created empty if it didn't exist. This happens regardless of whether you write anything afterward, which is why 'w' mode should only be used when you intend to replace content.

Q: Do I need to manually close a file if I use the with statement? No. The with statement's whole purpose is to close the file automatically when the indented block ends, whether it ends normally or because of an exception. Calling close() again afterward is harmless but unnecessary.

Q: What encoding does Python use when reading text files by default? Python uses the operating system's default encoding, which is UTF-8 on Linux and macOS, but can be different on Windows. For portable code, it's best practice to specify explicitly: open("file.txt", "r", encoding="utf-8").

Q: Can I read and write to the same file at the same time? Yes, using mode 'r+' (read and write, file must exist) or 'w+' (read and write, truncates first). These are less common and require careful use of seek() to control where the next read or write happens, since reading and writing share the same cursor position.

Q: What's the difference between a file "handle" and the file itself? The file itself is the data stored on disk. The file object (or "handle") returned by open() is an in-memory reference your program uses to interact with that data — like a remote control for the actual file, which is why you must close it when done.

Quick Revision

• File handling lets programs persist data beyond a single run by reading from and writing to disk.
• open(file_name, mode) returns a file object; common modes are 'r' (read), 'w' (write/overwrite), 'a' (append).
• 'w' mode erases existing content immediately on open; 'a' mode preserves it and adds to the end.
• read() returns the whole file as a string, readline() returns one line, readlines() returns a list of lines.
• Iterating directly over a file object (for line in file:) is memory-efficient for large files.
• The with statement automatically closes a file even if an exception occurs — always prefer it over manual open()/close().
• Text mode decodes bytes into characters (default UTF-8); binary mode ('rb'/'wb') passes raw bytes through unchanged.
• Mixing text mode with binary data (images, audio) causes UnicodeDecodeError or corrupted output.
• os.path.exists() checks whether a file exists before you try to read or delete it, avoiding FileNotFoundError.
• os.remove() deletes a file and raises an error if the file doesn't exist, so guard it with an existence check or try/except.
• Unflushed writes can be lost if a program crashes before the file is closed — closing (or using with) guarantees data reaches disk.
• Java's try-with-resources and C's manual fclose() solve the same problem as Python's with — reliably releasing a file handle.

Related Topics

Prerequisites: Variables and Data Types, Control Structures (loops and conditionals), Functions and Recursion

Related Topics: Exception Handling, Debugging and Testing, Operating Systems (file systems and permissions)

Next Topics: Debugging and Testing, Basics of OOP, Database Management Systems