A Beginner's Guide to File Management in Python

Okay, in the last post, we dove right into Python. We now know how to manually inject input data into the program every time it runs. Without a doubt, this is not the greatest technique to work with massive data sets. In reality, we want to save data so that we can read it later: we want to load and save data, exactly as in Microsoft Word. Look no further — in this post, we'll show you how to accomplish just that by handling files in Python.

 

Table of Contents about A Beginner's Guide to File Management in Python

Here’s what we are going to cover today:

• Python files... for beginners?


• Output and Input


• The Fundamentals of Python File Management


• Python CSV file reading


• The Code


• Using files as templates in Python


• Templating is now available.


• The principles behind templating


• Conclusion

 

Python files... for beginners?[ps2id id='Python files... for beginners?' target=''/]

Later in the course, many traditional textbooks address file management. Instead, we want to introduce them as soon as possible in our Python lessons. In the last article, you explored all of Python's core ideas. The traditional textbook goes away just as you begin to develop working scripts. You could expect to talk about functions and classes, which is OK.

However, we want you to start using Python as soon as feasible. Why not take advantage of the fact that it is not always essential to have a complete understanding of functions and classes in order to load and save a file? We'll use it to create a script that does something really useful.

Output and Input[ps2id id='Output and Input' target=''/]

All software and programs accept input and provide output. The program starts up and loads some data to process. The results were then presented elsewhere after more elaboration. Your application will have several ways for accepting input and producing output. For example, you may request input from the user at the command line. You may either print the results on the screen or save them to a file. You may, of course, import input data from a file as well.

You may think of your Python program as a line that communicates to files and/or the user over time. The arrows pointing to the line indicate that the program is receiving data, while the arrows pointing away from the line indicate that the program is sending data.At this point, you may consider a program to be something static. It starts, gets data there, puts data somewhere else, and then finishes. Some programs, such as the one we're working on right now, are just like that. But believe me when I say that every piece of software involves input/output logic. Even video games use what you type on the keyboard as input and display a new frame on your screen. They are constantly soliciting feedback and delivering data.

Your application may continue indefinitely, reading user input and writing the results to the screen.As can be seen, everything revolves on input and output. You will improve any software if you can simply collaborate with them. 

The Fundamentals of Python File Management[ps2id id='The Fundamentals of Python File Management' target=''/]

In Python, open a file.

The idea of opening a file exists in Python, as it does in many other programming languages. When you instruct your application to open a file, a variable is created that you may use to interact with the file. It allows you to read and write in it.

The modes

When opening a file, you need to specify how you would like to open the file. Are you going to read it? Or, are you going to write on it? You have several possibilities:

• r means you will be reading from the file


• w means you are going to write in the file, overriding the content


• a means that you are going to write in the file but after the existing content (append)

For all these options, the files must exist – Python won’t create a file on its own. Thus, if the file does not exist, it will throw an error. To avoid that, and create a new file if there is no existing one, you can add + to the mode. You will end up with r+, w+ and a+.

The code for opening a file in Python

Okay, but what is the code to open our beloved file? How do we specify the mode we talked about? Python is extremely intuitive and has an open() function to do that. Here’s a simple snippet.

file_variable = open(path_to_the_file, open_mode)

The path_to_the_file can be the file name in case the file is in the same folder as the script. Otherwise, it will look something like /subfolder/sub2/temp/file.txt. The open_mode is a string with the mode, so here’s a working example.

file_variable = open('input.txt', 'r+')

If you run a script with that code, it will create the file in the same folder as the script. Before continuing with our explanation, better creating a file to test what we are doing.

Simple Text files in Python

The simplest file in the world is probably a .txt file. It can contain only text, no fancy graphic or even text formatting styles. In the folder where you are working on Python scripts, create a new file named input.txt, you can do that with notepad. In the file, write a string of your liking – here’s an example.

This is my very first text file.

We are going to use this as input for a simple program.

Yay!

Of course, you can name the file whatever you want, we named it input just to be clear. Once we created the file, the most intuitive thing to do with it is opening and reading it.

Reading the file with Python

With a text file, you have two options. You can read the entire file immediately, or you can read it line by line. To do that, you need to call the read() function or the readline() function on the file variable you create when opening the file. The first will give you the content of the entire file, and the second the content of the first line that was not yet read.

You can also use .readlines() to read all lines and create a list containing the content of each. For all these three functions, you can easily print the results on the screen. Look at these examples.

Read the file as a whole

Here we use the read() function to read the entire file in a single line.

# Open the file in read mode plus

# We store the file handler in the variable 'file'

file = open('input.txt', 'r+')

# We store the file content in the variable 'file_content'

file_content = file.read()

print('The file content is:')

print(file_content)

And this will produce the following result.

C:\Users\aless\Desktop>python myscript.py

The file content is:

This is my very first text file.

We are going to use this as input for a simple program.

Yay!

Read the file line by line

Since we know that the file has four lines, we could use a for loop. Here’s an example of that.

# Open the file in read mode plus

# We store the file handler in the variable 'file'

file = open('input.txt', 'r+')

for i in range(4):

# We print the file, one line at a time

print(file.readline())

Here’s a much better way that uses the readlines() function. This is better because we don’t have to know the number of lines beforehand.

# Open the file in read mode plus

# We store the file handler in the variable 'file'

file = open('input.txt', 'r+')

# We can directly loop on the lines

for line in file.readlines():

# We print the line each time

print(line)

Both will result in the following output.

C:\Users\aless\Desktop>python myscript.py

This is my very first text file.

We are going to use this as input for a simple program.

Read More: Exceptions in Python: Managing Failures with Try-Except

Yay!

You can see that there is more space between lines. This is because each line ends with special characters, \n, which symbolizes a new line. The print function, however, adds another new line. As a result, we have two of them. You can see that by printing the list of lines with print(file.readlines()). It will give you the following output.

['This is my very first text file.\n', 'We are going to use this as input for a simple program.\n', '\n', 'Yay!']

Writing on a file

Now, we are going to use our w+ read mode to write in the file. Writing is much easier than reading because you basically print to the file. To do that, you only need one function: write(). Look at this example, that will also create the output.txt file.

# Open the file in write mode plus

# We store the file handler in the variable 'file'

file = open('output.txt', 'w+')

# Here we have a list of names

names = ['John', 'Amy', 'Mark', 'Bob', 'Jessica', 'Megan', 'Donald']

# We loop through the list...

for name in names:

# We write the name to the file

file.write(name)

# We write a new line so that each name is on a line

file.write('\n')

print('Finished!')

Now, we are going to have this content in output.txt.

John

Amy

Mark

Bob

Jessica

Megan

Donald

Pretty neat. You can combine these functions in many ways, for example, to keep an input file and an output file open simultaneously.

Closing the file

All good things come to an end. In the same way, whenever you open files in python you need to close them once you are done. This way the file is completely detached from your script and other applications access it. If you don’t close the file, you might end up corrupting it. Anyway, to close the file you simply call the close() function on the file.

file.close()

The better way

Since we are just starting, we do basic operations with files. In a complex program, however, you might end up with files that stay open for hundreds of lines of code. Even more, you may open a file within a selection construct (if/else) and close it later on. You might end up with some files that are left open, and this is a problem. So, we use the with construct, which takes a function like open() and a placeholder name. It is simpler than it might look.

with open('input.txt', 'r+') as file:

# Here we are inside the with construct

# We can use the 'file' variable

print(file.read())

# Now we are back outside the with construct

# So the file is automatically closed

# Using 'file' here will result in an error

This way, once we finish working inside the “with” construct, the file will close automatically. This is way better!

Python CSV file reading[ps2id id='Python CSV file reading' target=''/]

What exactly is CSV?[ps2id id='What exactly is CSV?' target=''/]

CSV stands for Comma-Separated Values and is a streamlined form of an excel file. Consider it a method for creating spreadsheets in plain text files. After all, a spreadsheet is a table, and each row in a table may be compared to a file line. To separate cells on the same row, use a comma. As a consequence, we've got a CSV file. It's simply that simple.

CSV files may be produced in Excel by selecting CSV from the Save As dialog box. You may also enter them into Notepad and save them as.csv.

Python's "csv" module is used to read CSV files.

We can do all of the heavy lifting using the tools we currently have. Python, on the other hand, has a module for dealing with CSV files. This is just a compilation of pre-existing Python code written by someone else that handles all of the complexities. Using the csv module, you can deal with CSV files in Python without worrying about the complexities. We start our script with import csv to notify it that it would need that module.

After that, we open a file and develop a custom CSV reader. You may loop around this variable to control how the file is read. At the conclusion of each loop cycle, one full row will be shown as a list. Here is one example.

The Code[ps2id id='The Code' target=''/]

# We specify that we need the csv module

import csv

# We open a CSV file like any other file

with open('variables.csv', 'r+') as file:

# We create a reader object

# Note that when creating the reader, we can optionally specify

# - delimiter, the character used to separate cells

# - quote character, the character used to enclose a single cells

# this is needed in case you want to include the delimiter

# character in the value of your cell

my_reader = csv.reader(file, delimiter=',', quotechar='"')

# We loop through the rows of the files

for row in my_reader:

print("ROW:")

# We loop through the cells in a row

# This is possible because 'row' is a list

for cell in row:

print(" " + cell)

# We print an empty line after finishing the cells

# So the output is clearer

print("")

Input and Output

Meanwhile, we have created a simple CSV file as an example, the variables.csv file. The content is fairly simple, and feel free to expand it as you wish.

John,Doe,22,American

Mark,Craig,39,British

Isabelle,Moretti,29,French

By running the script with this file, you will see this output.

C:\Users\aless\Desktop>python myscript.py

ROW:

John

Doe

22

American

ROW:

Mark

Craig

39

British

ROW:

Isabelle

Moretti

29

French

Using files as templates in Python[ps2id id='Using files as templates in Python' target=''/]

Templating is now available.[ps2id id='Templating is now available.' target=''/]

What precisely do we mean when we say "templating"? That is, the concept of creating a template. Assume you need to manufacture several duplicates of the same item with slight variations. For example, we might compose a greeting letter with the same content but different salutation and name for each recipient. Using CSVs and text files, we may create a CSV file with all of the variable items and a text file with the master template.

We can then run the script to produce a number of output files with the template created using the variables from the CSV file. This is beneficial in a wide range of IT applications. One example is networking, where you may end up configuring several devices with very similar settings. That is exactly what we are going to do today.

Read More: Exceptions in Python: Managing Failures with Try-Except

The principles behind templating[ps2id id='The principles behind templating' target=''/]

Each element in our CSV file will generate a new output file. This makes sense since each row may have numerous columns, and the columns are likely to be uniform throughout all rows. In each cell, you insert a variable object. For example, we might insert the name of the device for which the template is being built in the first cell of each row. In this way, we may finally see in a table all of our objects for which we need to build a file, as well as their variables. This is quite beneficial.

We may also deduce something crucial from this. Within a row, each cell might be recognized by a number. As a result, the first cell will be 0, the second will be 1, and so on. Remember it since we'll need it later.

#1 – Creating the variables

Our example script creates the configuration file for a Cisco switch. So, we need to define what are the variable configuration items we can specify in the CSV. For our example, each switch will be configured with Hostname, IP address, Subnet Mask, and default gateway. Therefore, each row will have four cells. Here’s an example of a CSV file respecting that concept.

HQ-AccessSwitch-01,192.168.5.10,255.255.255.0,192.168.5.1

HQ-AccessSwitch-02,192.168.5.11,255.255.255.0,192.168.5.1

HQ-AccessSwitch-03,192.168.5.12,255.255.255.0,192.168.5.1

Brach-AccessSwitch-01,192.168.25.10,255.255.255.0,192.168.25.1

Brach-AccessSwitch-02,192.168.25.11,255.255.255.0,192.168.25.1

#2 – Creating the template file

Now, we need to create the template file. It will include some placeholders that indicate “Put here the first cell, here the second and there the third”, or something like that. You can create placeholders the way you like, but here we decided to stick with a syntax similar to Python.

So, in our file, the placeholder will be the number of cells within curly brackets. If we want to say “Put here the content of the first cell”, we are going to use 0. If we want the second, we are going to use 1 and so on. Here we have a template that creates a valid configuration for a Cisco switch.

hostname 0

!

interface Vlan1

ip address 1 2

no shut

exit

!

!

ip default-gateway 3

!

We can store this in the template.txt file, and now we are ready for scripting.

#3 – The Script itself

Working with files in Python is easy. Even the following script is easy, take a moment to look at it.

# We specify that we need the csv module

import csv

# First, we open the template file

with open('template.txt', 'r+') as template_file:

# We read the template and store it in a variable

template = template_file.read()

# We open the list of variables (CSV)

with open('variables.csv', 'r+') as variables_file:

# We exract the CSV table with the csv module

variables = csv.reader(variables_file, delimiter=',', quotechar='"')

# We loop through the rows in the variables table

# Each row represents an output file to be created

for row in variables:

# We create a txt file with the name of the first cell

# In this example, it is going to be the device's name

with open(row[0] + '.txt', 'w+') as output_file:

# We copy the template to a new variable

# so that we don't alter the original template

dirty_template = template

# we loop through all cells of our row

for n, cell in enumerate(row):

# We are going to look for the cell number

# within curly brackets, e.g. 0, 1 etc.

search_for = '' + str(n) + ''

# We replace any occurency of the placeholder

# with the value of the celll

# and we update the dirty template so that it includes

# our last modifications

dirty_template = dirty_template.replace(search_for, cell)

# After all the replacements, we save to the file

output_file.write(dirty_template)

# Here the output file is closed

# we inform the user and move to the next file

print("File '" + row[0] + "' created")

Use the comments to understand the script, then read on for an high-level explanation.

A quick explanation

Nested with constructs, copying variables together, dynamically create a file… this might look complex, but it isn’t. First, we open the template file and we store the content in the form of a string. Then, we open the file containing the variables and we start to loop for all the rows. Since each row represents a different set of data to use for compiling the template, each row will result in a new output file.

When looping through the rows, we start by opening (creating) a new file. To better understand what is what, we name it with the content of the first cell (row[0]). Then, we copy the template to a variable that is local to this part of the code. We are going to alter it, while the template is not modified. Thus, we can re-use the template for the next row. For each cell, we use the replace() function to replace the placeholder with its value. Once we finish, we write the result to the file.

Note that we used the enumerate() function when looping on cells. Looping on cells without that will put in cell the value of the cell everytime, until we pass through all the cells. With enumerate, we also put the number of the cell within a row in the n variable. This way, we can use this value to create the search_for string.

Read More: Exceptions in Python: Managing Failures with Try-Except

Conclusion[ps2id id='Conclusion' target=''/]

Isn't it great to be able to deal with files in Python? With this knowledge and the focus on templating, you can now construct a script that performs something useful. If you work in IT, like me, you'll appreciate how much time this script will save you!

If Our Method Resolve Your Problem Consider To Share This Post, You can help more People Facing This Problem and also, if you want, you can Subscribe at Our Youtube Channel as Well!

Was it easy for you to work with files? Did you understand it right away? What is your most typical method of using files? Simply let me know by leaving a comment!

https://www.techguruhub.net/guide-file-management-in-python/?feed_id=112283&_unique_id=6411f9746a254