Python File Handling

Posted on Edited on In , Symbols count in article: 11k Reading time ≈ 10 mins.

Python Tutorial

Reference

W3Schools
Reading and Writing Files

Python File Open

File handling is an important part of any web application.

Python has several functions for creating, reading, updating, and deleting files.

File Handling

The key function for working with files in Python is the open() function.

The open() function takes two parameters; filename, and mode.

There are four different methods (modes) for opening a file:

"r" - Read - Default value. Opens a file for reading, error if the file does not exist.

"a" - Append - Opens a file for appending, creates the file if it does not exist.

"w" - Write - Opens a file for writing, creates the file if it does not exist.

"x" - Create - Creates the specified file, returns an error if the file exists.

'r' open for reading (default)
'w' open for writing, truncating the file first
'x' create a new file and open it for writing
'a' open for writing, appending to the end of the file if it exists
'b' binary mode
't' text mode (default)
'+' open a disk file for updating (reading and writing)
'U' universal newline mode (deprecated)

In addition you can specify if the file should be handled as binary or text mode

"t" - Text - Default value. Text mode.

"b" - Binary - Binary mode (e.g. images).

Syntax

To open a file for reading it is enough to specify the name of the file:

1
f = open("demofile.txt")

The code above is the same as:

1
f = open("demofile.txt", "rt")

Because "r" for read, and "t" for text are the default values, you do not need to specify them.

Note: Make sure the file exists, or else you will get an error.

Type
1
2
3
4
5
6
f = open("demofile.txt")

type(f)
# _io.TextIOWrapper
type(f.read())
# str

Open a File on the Server

Assume we have the following file, located in the same folder as Python:

Shell demofile.txt
1
2
3
Hello! Welcome to demofile.txt
This file is for testing purposes.
Good Luck!

To open the file, use the built-in open() function.

The open() function returns a file object, which has a read() method for reading the content of the file:

Python
1
2
3
f = open("demofile.txt", "r")
f.read()
# 'Hello! Welcome to demofile.txt\nThis file is for testing purposes.\nGood Luck!\n'

.read()

1
2
3
4
5
6
7
Signature: f.read(size=-1, /)
Docstring:
Read at most n characters from stream.

Read from underlying buffer until we have n characters or we hit EOF.
If n is negative or omitted, read until EOF.
Type:      builtin_function_or_method

Read line by line

Best choice:

Python
1
2
3
f = open('demofile.txt', 'r')
for line in f.read().splitlines():
    print(line)

Optional solutions:

Python
1
2
3
f = open('demofile.txt', 'r')
for line in f:
    print(line.strip())
Python
1
2
3
4
f.close()
f = open('demofile.txt', 'r')
for line in f.readlines():
    print(line.strip())

Read Only Parts of the File

By default the read() method returns the whole text, but you can also specify how many characters you want to return:

Python
1
2
3
f = open("demofile.txt", "r")
f.read(5) # Return the 5 first characters of the file:
# 'Hello'

.readline()

1
2
3
4
5
6
Signature: f.readline(size=-1, /)
Docstring:
Read until newline or EOF.

Returns an empty string if EOF is hit immediately.
Type:      builtin_function_or_method

You can return one line by using the readline() method:

Python
1
2
3
f = open("demofile.txt", "r")
print(f.readline()) # Read one line of the file:
# Hello! Welcome to demofile.txt

By calling readline() two times, you can read the two first lines:

Python
1
2
3
4
5
f = open("demofile.txt", "r")
print(f.readline()) # Read one line of the file:
# Hello! Welcome to demofile.txt
print(f.readline())
# This file is for testing purposes.

By looping through the lines of the file, you can read the whole file, line by line:

Python
1
2
3
4
5
6
7
8
f = open("demofile.txt", "r")
for lines in f:
    print(lines)
# Hello! Welcome to demofile.txt

# This file is for testing purposes.

# Good Luck!

.readlines()

1
2
3
4
5
6
7
8
Signature: f.readlines(hint=-1, /)
Docstring:
Return a list of lines from the stream.

hint can be specified to control the number of lines read: no more
lines will be read if the total size (in bytes/characters) of all
lines so far exceeds hint.
Type:      builtin_function_or_method

Close Files

Warning: Calling f.write() without using the with keyword or calling f.close() might result in the arguments of f.write() not being completely written to the disk, even if the program exits successfully.

It is a good practice to always close the file when you are done with it.

Close the file when you are finish with it:

Python
1
2
3
4
f = open("demofile.txt", "r").splitlines()
print(f.readline())
# Hello! Welcome to demofile.txt
f.close()

Note: You should ALWAYS close your files, in some cases, due to buffering, changes made to a file may not show until you close the file.

With method:

Close the file automatically:

Python
1
2
3
4
5
with open("demofile.txt", "r") as f:
    print(f.read().splitlines())
# Hello! Welcome to demofile.txt
# This file is for testing purposes.
# Good Luck!

Why you need to close the file

Shell
1
echo 'Hello World' > sample_file.txt
Python
1
2
3
4
files = []
for i in range(100000):
    files.append(open('sample_file.txt', 'r'))
    print(i)

The I/O limitation is differ in different computers. In my computer, I cannot open more than 10490 files. If you don’t close it, the file will keep opening.

1
2
3
4
10490
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
OSError: [Errno 24] Too many open files: 'sample_file.txt'

Linux Word Count

CLI
1
wc <file>
CLI
1
2
3
4
5
6
7
8
9
10
11
man wc
'''
-c      The number of bytes in each input file is written to the standard output.  This will cancel out any prior usage of the -m option.

-l      The number of lines in each input file is written to the standard output.

-m      The number of characters in each input file is written to the standard output.  If the current locale does not support multibyte characters, this is equivalent to the -c option.  This will cancel out any
        prior usage of the -c option.

-w      The number of words in each input file is written to the standard output.
'''

-w is for counting the number of words.

Python File Write

To write to an existing file, you must add a parameter to the open() function:

"a" - Append - will append to the end of the file

"w" - Write - will overwrite any existing content

Append:
1
2
3
4
5
6
7
8
9
with open("demofile.txt", "a") as f: # Open the file "demofile.txt" and append content to the file:
    f.write("Now the file has more content!")

with open("demofile.txt", "r") as f: # Open the file "demofile.txt" and read the content:
    print(f.read())
# Hello! Welcome to demofile.txt
# This file is for testing purposes.
# Good Luck!
# Now the file has more content!
Overwrite:
1
2
3
4
5
6
with open("demofile.txt", "w") as f: # Open the file "demofile.txt" and overwrite content to the file:
    f.write("Woops! I have deleted the content!")

with open("demofile.txt", "r") as f: # Open the file "demofile.txt" and read the content:
    print(f.read())
# Woops! I have deleted the content!

Note: the “w” method will overwrite the entire file.

.write()

1
2
3
4
5
6
Signature: f.write(text, /)
Docstring:
Write string to stream.
Returns the number of characters written (which is always equal to
the length of the string).
Type:      builtin_function_or_method

.writelines()

1
2
3
4
5
6
7
Signature: f.writelines(lines, /)
Docstring:
Write a list of lines to stream.

Line separators are not added, so it is usual for each of the
lines provided to have a line separator at the end.
Type:      builtin_function_or_method

Python File Replace

Replace One Line

Syntax

Python
1
2
3
4
5
6
7
8
import os
import re

with open(<filename>, 'r+') as f:
    content = f.read()
    f.seek(0)
    f.write(re.sub(<repl, string, content>))
    f.truncate
Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
import os
import re

help(f.seek)

'''
Help on built-in function seek:

seek(cookie, whence=0, /) method of _io.TextIOWrapper instance
    Change stream position.
    
    Change the stream position to the given byte offset. The offset is
    interpreted relative to the position indicated by whence.  Values
    for whence are:
    
    * 0 -- start of stream (the default); offset should be zero or positive
    * 1 -- current stream position; offset may be negative
    * 2 -- end of stream; offset is usually negative
    
    Return the new absolute position.
'''
Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
import os
import re

help(f.seek)

'''
Help on built-in function truncate:

truncate(pos=None, /) method of _io.TextIOWrapper instance
    Truncate file to size bytes.
    
    File pointer is left unchanged.  Size defaults to the current IO
    position as reported by tell().  Returns the new size.
'''

Example

Python
1
2
3
4
5
6
7
8
9
10
import os
import re

os.chdir('/Users/zacks/Desktop/ZacksAmber.github.io/source/_posts')

with open('/Users/zacks/Desktop/ZacksAmber.github.io/source/_posts/notes-catalog.md', 'r+') as f:
    catalogContent = f.read()
    f.seek(0)
    f.write(re.sub('Total Words Count of All Posts: .*\n', 'Total Words Count of All Posts: ' + postsWordsCount + '\n', catalogContent))
    f.truncate()

Replace Multiple Lines

Syntax

Python
1
2
3
4
5
6
7
8
9
import os
import re

with open(<filename>, 'r') as f:
    content = f.read()

with open(<filename>, 'w') as f:
    for i in <contentMatched>:
        ...

Example

Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
import os
import re

# Store all posts name in posts
postsPath = '/Users/zacks/Desktop/ZacksAmber.github.io/source/_posts'
os.chdir(postsPath)
posts = os.listdir(postsPath)
posts.remove('.ds_store') # remove non-.md file
posts.remove('untitled.code-workspace') # remove non-.md file
posts.sort() # sort the posts

for post in posts:
    newPost = post.split('.')
    newPost = newPost[0] + '_new.' + newPost[1]
    # Read post file as postFile
    with open(post, 'r') as postFile:
        # Get post content from postFile
        postContent = postFile.read()
        postMatched = re.findall('##+.*\n.', postContent) # e.g. '### JSON in Python\nP'

    # Write post file as postFile
    '''
    with open('tmp1.md', 'w') as postFile:
        for i in postMatched:
            subtile = i.split('\n')
            subtitle = subtile[0] + '\n\n' + subtile[1]
            postFile.write(re.sub(subtitle, i, postContent))
    '''
    
    if len(postMatched) != 0:
        # Write post file as postFile
        with open(post, 'w') as postFile:
            for i in postMatched:
                subtile = i.split('\n')
                subtitle = subtile[0] + '\n\n' + subtile[1]
                postContent = postContent.replace(i, subtitle)
            postFile.write(postContent)

Python Create File

To create a new file in Python, use the open() method, with one of the following parameters:

"x" - Create - will create a file, returns an error if the file exist.

"a" - Append - will create a file if the specified file does not exist.

"w" - Write - will create a file if the specified file does not exist.

1
2
f = open("demofile.txt", "x")
# FileExistsError: [Errno 17] File exists: 'demofile.txt'
1
f = open("myfile.txt", "a")
1
f = open("myfile2.txt", "w")

Python Delete File

Delete a File

To delete a file, you must import the OS module, and run its os.remove() function:

Remove the file “demofile.txt”:

1
2
import os
os.remove("demofile.txt")

Check if File exist:

To avoid getting an error, you might want to check if the file exists before you try to delete it:

Check if file exists, then delete it:
1
2
3
4
5
6
7
8
9
import os

os.path.exists("myfile.txt")
# True

if os.path.exists("myfile.txt"):
    os.remove("myfile.txt")
else:
    print("The file does not exist")

Delete Folder

To delete an entire folder, use the os.rmdir() method:

1
2
import os
os.rmdir("myfolder")

Note: You can only remove empty folders.

For more information, click here

One More Thing

Python Algorithms - Words: 2,640

Python Crawler - Words: 1,663

Python Data Science - Words: 4,551

Python Django - Words: 2,409

Python File Handling - Words: 1,533

Python Flask - Words: 874

Python LeetCode - Words: 9

Python Machine Learning - Words: 5,532

Python MongoDB - Words: 32

Python MySQL - Words: 1,655

Python OS - Words: 707

Python plotly - Words: 6,649

Python Quantitative Trading - Words: 353

Python Tutorial - Words: 25,451

Python Unit Testing - Words: 68