Files in Python , Exception handling

While a program is running, its data is in main memory. When the program ends, or the computer shuts down, data in memory disappears. To store data permanently, you have to put it in a file. Files are usually stored on a secondary storage device(hard disk, pen drive, DVD,CD etc). 

When there are a large number of files, they are often organized into directories (also called “folders”). Each file is identified by a unique name, or a combination of a file name and a directory name. 

By reading and writing files, programs can exchange information with each other and generate printable formats like PDF. Working with files is a lot like working with books. To use a book, you have to open it. When you’re done, you have to close it. While the book is open, you can either write in it or read from it. In either case, you know where you are in the book. Most of the time, you read the whole book in its natural order, but you can also skip around. All of this applies to files as well. 

To open a file, you specify its name and indicate whether you want to read or write. Opening a file creates a file object. In this example, the variable ‘f ‘ refers to the new file object.

>>> f = open("test.dat","w")

This will create a file test.dat in the current directory. If the file already exists, it will overwrite.

The open function takes two arguments. The first is the name of the file, and the second is the mode. Mode "w" means that we are opening the file for writing. The following are common file opening modes.

r- for reading
w-writing
a-appending
r+- reading and writing
a+-appending and reading
w+-writing and reading
rb-read a binary file
wb-writing a binary file
rb+-reading and writing a binary file
wb+-writing and reading a binary file
ab+-appending or reading a binary file

>>> print f
<open file 't.dat', mode 'w' at 0x018B4DE0>

When we print the file object, we see the name of the file, the mode, and the location of the object.
Note: The open command will open a file in the current directory. If we want to open a file in some other directory, we have to specify complete path to the file.
f=open(“c:\\cek\\test.dat”,”r”)

Writing data into file
To put data in the file we invoke the write method on the file object after opening the file in write mode:
>>> f = open("test.dat","w")
>>> f.write("Python Programming\n")
>>> f.write("It is great")
>>> f.close()
Closing the file tells the system that we are done writing and makes the file available for reading:
Now test the file content test.dat by opening it any editor(notepad(windows),gedit,vi etc(linux)).

The file method write expects a string as an argument. Therefore, other types of data, such as integers or floating-point numbers, must first be converted to strings before being written to an output file. In Python, the values of most data types can be converted to strings by using the str function. The resulting strings are then written to a file with a space or a newline as a separator character.

The following code will write numbers 1-10 line by line in the file num.dat

f=open("num.dat","w")
for i in range(1,11):
   f.write(str(i)+"\n")
f.close()

The argument of write has to be a string, so if we want to put other values in a file, we have to convert them to strings first. The easiest way to do that is with the str function:

An alternative is to use the format operator %.

Example
>>> cars = 52
>>> "In July we sold %d cars." % cars
’In July we sold 52 cars.’
>>> "In %d days we made %f million %s." % (34,6.1,’dollars’)
’In 34 days we made 6.100000 million dollars.’

When the file is opened in write(w) mode and if the file already exist , write command will overwrite the contents.If you want to add contents to an existing file, the file must be opened in append(a)mode. The following command will open the ‘test.dat’ file and append the new contents.

>>>f=open(‘test.dat’,”a”)
>>>f.write(“this is the new content”)
>>>f.close()

The method writelines() writes a sequence of strings to the file. The sequence can be any iterable object producing strings, typically a list of strings. There is no return value.

>>>Lst=[“this is a test\n”,”on writelines method in python\n”,”file handling”]

>>>f=open(‘test.dat’,’w’)
>>>f.writelines(Lst)

Reading data from a file
For reading data from a file, first open the file in read mode
>>>f=open(“test.dat”,”r”)
If we try to open a file that doesn’t exist, we get an error:
>>> f = open("test.cat","r")
IOError: [Errno 2] No such file or directory: ’test.cat’
The read() method will read the entire contents of the file as string.
>>> l=f.read()
>>> print l
Python Programming
It is great

Note that once the contents are read with read() method, the file pointer is moved to the end of file. The read() method will then return an empty string. If you want to read it again move the file pointer to the beginning of the file using seek() command.

All reading and writing functions discussed till now, work sequentially in the file. To access the contents of file randomly - seek and tell methods are used.

tell() method returns an integer giving the current position of object in the file. The integer returned specifies the number of bytes from the beginning of the file till the current position of file object.It's syntax is
fileobject.tell()

seek() method can be used to position the file object at particular place in the file. It's syntax is :
fileobject.seek(offset [, from_what])

here offset is used to calculate the position of fileobject in the file in bytes. Offset is added to from_what (reference point) to get the position. Following is the list of from_what values:

Value reference point

0 beginning of the file
1 current position of file
2 end of file

default value of from_what is 0, i.e. beginning of the file.
>>>f.seek(0)
This will read 5 bytes from the file and the file pointer is advanced to the 6’th byte.
>>>f.read(5)
Pytho

A text file is a file that contains printable characters and whitespace, organized into lines separated by newline characters. Since Python is specifically designed to process text files, it provides methods that make the job easy.

The readline() command can be used to read line by line from a file

>>> f = open("test.dat","w")
>>> f.write("line one\nline two\nline three\n")
>>> f.close()

The readline method reads all the characters up to and including the next newline character:

>>> f = open("test.dat","r")
>>> print f.readline()
line one
>>>

The file pointer will be advanced to next line after reading each line.
f.readlines() will read all lines into a list.

we can also use a for loop which will read file line by line

f=open(“test.dat”,”r”)
for l in f:
   print l

with statement

You can also work with file objects using the with statement. It is designed to provide much cleaner syntax and exceptions handling when you are working with code. One bonus of using this method is that any files opened will be closed automatically after you are done. 

with open("file1.dat") as f:
for line in f:
   print line,

*****************************************************************

Pickling
In order to put values into a file, you have to convert them to strings. You have already seen how to do that with str:
>>> f.write (str(12.3))
>>> f.write (str([1,2,3]))

The problem is that when you read the value back, you get a string. The original type information has been lost. In fact, you can’t even tell where one value ends and the next begins:

>>> f.readline()
’12.3[1, 2, 3]’

The solution is pickling, so called because it “preserves” data structures. The pickle module contains the necessary commands. To use it, import pickle and then open the file in the usual way:

>>> import pickle
>>> f = open("test.dat","w")

To store a data structure, use the dump method and then close the file in the usual way:

>>> pickle.dump(12.3, f)
>>> pickle.dump([1,2,3], f)
>>> f.close()

Then we can open the file for reading and load the data structures we dumped:

>>> f = open("test.pck","r")
>>> x = pickle.load(f)
>>> x
12.3
>>> type(x)
<type ’float’>
>>> y = pickle.load(f)
>>> y
[1, 2, 3]
>>> type(y)
<type ’list’>
Each time we invoke load, we get a single value from the file, complete with its original type.

***********************************************************

Exception Handling
Whenever a runtime error occurs, it creates an exception. Usually, the program stops and Python prints an error message.
Examples of different type of exceptions

Zero division error-division by zero
>>> 2/0
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
2/0
ZeroDivisionError: integer division or modulo by zero

indexError-index out of range
>>> a=[10,20,30]
>>> print a[3]
Traceback (most recent call last):
File "<pyshell#3>", line 1, in <module>
print a[3]
IndexError: list index out of range

>>> a={'a':10,'b':20}
>>> a['c']
Key error-use of non existent key
Traceback (most recent call last):
File "<pyshell#5>", line 1, in <module>
a['c']
KeyError: 'c'
IO error-opening a non existing file
>>> f=open("temp.data","r")
Traceback (most recent call last):
File "<pyshell#6>", line 1, in <module>
f=open("temp.data","r")
IOError: [Errno 2] No such file or directory: 'temp.data'

Note that Python prints a traceback of where the program was and also print the error message with description.

Sometimes we want to execute an operation that could cause an exception, but we don’t want the program to stop. We can handle the exception using the try and except statements in Python

For example, we might prompt the user for the name of a file and then try to open it. If the file doesn’t exist, we don’t want the program to crash; we want to handle the exception:

filename = raw_input(’Enter a file name: ’)

try:
  f = open (filename, "r")
except IOError:
  print ’There is no file named’, filename

The try statement executes the statements in the first block. If no exceptions occur, it ignores the except statement. If an exception of type IOError occurs, it executes the statements in the except branch and then continues.

You can write a function exist() which will return true if the file exist or False otherwise.

def exists(filename):
try:
  f = open(filename)
  f.close()
  return True
except IOError:
  return False

Note: We can also write multiple except blocks to handle different type of exceptions.

We can also raise an exceptions in a program by raise command. In this case the program stops after displaying the error message.try except block can also be used to handle the raise exceptions.

Example
x=input("enter x")
y=input("enter y")
if y==0:
   raise ValueError,'bad input value zero'
else:
  print x/y 
*****************************************************************************

Sample Programs
The following program will copy file1.dat to file2.dat
f1 = open("file1.dat", "r")
f2 = open("file2.dat", "w")
while True:
    l=f1.readline()
   if l=="":
       break
f2.write(l)
f1.close()
f2.close()
print 'File is copied'

The following program will copy file1.dat to file2.dat after removing lines starting with # (comment lines in python)
f1=open("file1.dat",'r')
f2=open("file2.dat",'w')
while True:
     l=f1.readline()
    if l=="":
       break
   if l[0]!='#':
     f2.write(l)
f1.close()
f2.close()
print 'File is copied'
The following program will copy file1.dat to file2.dat after removing blank lines
( university question)
f1 = open("file1.dat", "r")
f2 = open("file2.dat", "w")
lst=f1.readlines() # reading lines to a list
for l in lst:
   if l !='\n':
   f2.write(text) # writing non empty lines to new file
f1.close()
f2.close()
print "File copied"
Input n numbers and store the cube of each number in a file

f=open("cube.dat","w")
n=input("Enter n..")
print "Enter ",n ," numbers"
for i in range(0,n):
   x=input()
   x=x**3
   f.write(str(x)+"\n")
f.close()
print "file created with cube of numbers"
program that reads a file and writes out a file with the lines in reversed order.
( university question)
f1 = open("file1.dat", "r")
f2 = open("file2.dat", "w")
lst=f1.readlines() # read the entire contents into a list
rlst=reversed(lst) #reversing the list
f2.writelines(rlst) # writing each line in the new file
f1.close()
f2.close()

We can also do the above program by reversing the file object
f1 = open("file1.dat", "r")
f2 = open("file3.dat", "w")
rf1=reversed(list(f1)) # reversing the file object after converting it into a list sequence
for line in rf1:
   f2.write(line) # writing each line in the new file
f1.close()
f2.close()
print "File copied"

Read and print a file with all punctuation removed.( university question)
import string
f1 = open("file1.dat", "r") #create the file file1.dat with punctuations
for line in f1: # read each line
   print line.translate(None,string.punctuation), #translate punctuations to none
f1.close()
program ro read a text file and print each word and its length in sorted order
f=open("file1.dat","r")
str=f.read() #reading the file contents into a string
str=str.replace("\n"," ") #replace all newline with spaces
words=str.split(" ") #split the string into words
words.sort()
print "words and length"
for w in words:
  print w,len(w)
f.close()
Create a file with 10 integer numbers and finding the average of odd numbers
#writing 10 numbers into a file
f=open("num.dat","w")
print "Enter 10 integers.."
for i in range(10):
  n=input() # if you are using raw_input no need of str conversion
f.write(str(n)+"\n") # writing numbers line by line
f.close()

#reading numbers and finding average of odd numbers
f=open("num.dat","r")
s=0
c=0
for n in f:
  num=int(n) #converting string n to int num
  if num%2 !=0:
    s=s+num # finding sum of odd numbers
    c=c+1 # finding the count of odd numbers
print "the sum of odd numbers=",s,"average=",s/c
f.close()

Write a program to read numbers stored in one file and store the sorted numbers in another file after deleting duplicates.(university question)
f1=open("file1.dat","r") # numbers are stored line by line
lst=f1.readlines() # read the numbers into a list
nlst=list(set(lst)) # removing duplicates easy method is to make a set
nlst.sort() # sorting the list
f2=open("file2.dat","w") #writing into a new file
f2.writelines(nlst)
f1.close()
f2.close()
print "sorted file is created"
Finding the count of four letter words in a file ( University Question)
f=open("data.txt","r")
st=f.read()
stn=st.replace("\n"," ")
words=stn.split(" ")
c=0
for w in words:
if len(w)==4:
  print w
  c=c+1
print "Count of 4 letter words=",c
f.close()
Write a python program to create a file with 10 integer numbers. Read this file and display the square of each number. ( university question)
f=open("num.dat","w")
print "Enter 10 numbers..."
for i in range(10):
  x=input()
  f.write(str(x)+"\n")
f.close()
f=open("num.dat","r")
print "Numbers and its square.."
for i in f:
  n=int(i)
  print n,n**2
f.close()

Reading numbers from a file and storing even and odd numbers in two separate files
f1=open("file1.txt","r")
f2=open("odd.txt","w")
f3=open("even.txt","w")
l=f1.readlines()
for i  in l:
  n=int(i)
  if n%2==0:
     f3.write(i)
 else:
    f2.write(i)
f1.close()
f2.close()
f3.close()

print 'Files are created...'

Reading numbers from a file and storing positive and negative numbers in two separate files
 ( university question)
f1=open("num.txt","r") # numbers are stored line by line
lst=f1.readlines() # read the numbers into a list
print lst
p=[]
n=[]
for i in lst: # seperating postive and negative numbers
  if int(i)<0:
     n.append(i)
  else:
    p.append(i)
f2=open("positive.txt","w") #writing into a new files
f3=open("negative.txt","w")
f2.writelines(p)
f3.writelines(n)
f1.close()
f2.close()
f3.close()
print "Files created"

Print prime numbers from a set of numbers stored in a file
def primetest(n):
   prime=True
   i=2
   while i<=n/2:
      if n%i==0:
          prime=False
          break
      i=i+1
   return prime

pf=open("prime.dat","w")
while True:
   n=f.readline()
   if n=="":
       break
   n=int(n)
   if primetest(n):
     print(n)
pf.close()

Read name and mark in two subjects of a student and store it in a file. Display the name and sum of marks.( use pickling)
import pickle
f=open("stud.dat","w")
name=raw_input("Enter the name...")
m1=input("mark in subject1..")
m2=input("mark in subject2...")
pickle.dump(name,f)
pickle.dump(m1,f)
pickle.dump(m2,f)
f.close()
f=open("stud.dat","r")
namef=pickle.load(f)
m1f=pickle.load(f)
m2f=pickle.load(f)
print "name=",namef,"total mark=",m1f+m2f
f.close()
Python program to read two matrices from two files, find the sum and display the resultant matrix.Assume the first line of the input file represent the order of the matrix in a comma separated format and the remaining lines represent the rows of the matrix in a comma separated format.(university question)
f1=open("mat1.dat","r") # open the first matrix file mat1.dat
f2=open("mat2.dat","r") # open the second matrix file mat2.dat
first1=f1.readline() # read the first lines from both the files
first2=f2.readline() #and check the dimension
dim1=first1.split(",")
m1=int(dim1[0])
n1=int(dim1[1])
dim2=first2.split(",")
m2=int(dim2[0])
n2=int(dim2[1])
if m1 != m2 or n1 != n2:
  print "Dimension doesnt match cant add"
  exit
else:
   print "Sum of two matrix" # reading each row and finding the sum
for i in range(m1):
 r1=f1.readline()
 r2=f2.readline()
 r1=r1.split(",")
 r2=r2.split(",")
 r3=[]
 for j in range(n1):
   r3.append(int(r1[j])+int(r2[j]))
 print r3
f1.close()
f2.close()

Write a Python program to store lines of text into a file.Read the file and display only the palindrome words in the file.(university question)
#creating file
f=open("file.data","w")
while True:
  ln=raw_input("Enter a line of text...type quit to exit\n")
  if ln=='quit':
     break
  f.writelines(ln+'\n')
f.close()

#reading and printing palindrome words
f=open("file.data","r")
lst=f.read()
lst=lst.replace("\n"," ")
words=lst.split(" ")
print "Palindrome words"
for w in words:
  if w==w[::-1]:
     print w
f.close()
Write a Python program to create a text file. Read the contents of the file, encrypt every character in the file with a distance of 3 and write it to a new file.
Eg:yak is encrypted as bdn.( university question)

f=open("text.dat","r")
alpha="abcdefghijklmnopqrstuvwxyz"
ns=""
s=f.read()
for c in s:
        if c in alpha:
               c=ord(c)+3
               if c > 122:
                   c=c-26   
               c=chr(c)
        ns=ns+c
nf=open("textnw.dat","w")
nf.write(ns)
f.close()
nf.close()
print "FILE ENCRYPTED..."

Try the following Programs in the lab
1.Input list of strings and create a text file with this sequence.
2.Copy one file to another
3.Copy one python program file to another after removing the single line comments
( lines starting with #)
4.Read and print a file with line numbers of each line.
5.Read and print a file with all punctuation removed.( university question)
6.Write a python program to create a file with 10 integer numbers. Read this file and display the square of each number. (University question)
7.Write a program that reads a file and writes out a file with the lines in reversed order.( university question-refer example)
8.Write a python program to read a text file, copy the content to another file after removing the blank lines.( university question)
9.Count the number of lines, number of words and number of characters in a file.(refer example)
10.Program to read a text file and print the words and its length in sorted order.( refer example)
11.Write a Python program that opens a file for input and prints the count of four letter words.( university question..)
12.write a program to read numbers stored in one file and store the sorted numbers in another file after deleting duplicates.(university question-refer example)
13.Write rollnumber(integer) name(string) and attendance percentage(float) of n students into a file. Read the file and show the list of students having attendance shortage(less than 75%).(use pickling)
14.Read a file and display the lines with number of words in each line. Use a function exist() which will handle an exception if the file does not exist.
15.Write a python program to read numbers from a file named num.txt.Write all positive numbers from num.txt to file named positive.txt and all negative numbers to negative.txt.(university question)
16.Write a python program to read two matrices from two files, find the sum and display the resultant matrix.Assume the first line of the input file represent the order of the matrix in a comma separated format and the remaining lines represent the rows of the matrix in a comma separated format.(university question)
17.Write a Python program to store lines of text into a file.Read the file and display only the palindrome words in the file.(university question)
18.Write a python program to input list of n numbers. Calculate cube of each number and store it in a file.( university question)
19.Write a Python program to create a text file. Read the contents of the file, encrypt every character in the file with a distance of 3 and write it to a new file.
Eg:yak is encrypted as bdn.( university question)