How to Copy a File in Python
When it comes to using Python to copy files, there are two main ways: using the
shutil module or the
os module. All of the
os methods we show here are methods that allow us to execute shell commands from our Python code, which we’ll use to execute the
copy command (Windows) or the
cp command (Unix).
You’ll notice that many of these methods, in both the
shutil module and the
os module, have very similar functionality (which shouldn’t be surprising), but each varies in functionality from each other very slightly, which I’ll explain as well.
Copying Files with the shutil Module
The shutil module offers several high level methods to copy files. Here below are the main ones:
This method copies the content of one file into another file. The destination provided to it must be a writable file, and have a different name than the source file. If the names are the same then it will generate an error. If the destination file already exists, it will be replaced with the newly copied file.
The syntax for this method is:
shutil.copyfile(src_file, dest_file, *, follow_symlinks=True)
For example, the following code will copy a file named “file1.txt” into a file named “file2.txt”:
import shutil shutil.copyfile('file1.txt', 'file2.txt')
One interesting and potentially useful feature of
shutil.copyfile is the
follow_symlinks Boolean argument. If it is set to
False, and the source file is a symbolic link, then instead of copying the file, a new symbolic link will be created.
This method is very similar to
copyfile, with the main difference being that in addition to copying the content of the source file, it goes one step further and also copies the file’s file system permissions. Copying file permissions is not a trivial task in most programming languages, so this is a nice feature to have.
The syntax is as follows:
shutil.copy(src_file, dest_file, *, follow_symlinks=True)
Each of these parameters are the same as in the
copyfile method. For example, the following code will copy “file1.txt” into “file3.txt”.
import shutil shutil.copy('file1.txt', 'file3.txt')
As with the previous methods,
copy2 method is identical to the
copy method, but in addition to copying the file contents it also attempts to preserve all the source file’s metadata. If the platform doesn’t allow for full metadata saving, then
copy2 doesn’t return failure and it will just preserve any metadata it can.
The syntax is as follows:
shutil.copy2(src_file, dest_file, *, follow_symlinks=True)
Again, these parameters are the same as in the previous commands we’ve mentioned so far.
For example, the following code will copy “file1.txt” into “file4.txt”, as well as preserve the metadata of the original file, “file1.txt”.
import shutil shutil.copy2('file1.txt', 'file4.txt')
$ python copy-files.py $ ls -l total 32 -rw-r--r-- 1 scott staff 91 Oct 27 11:26 copy-files.py -rw-r--r-- 1 scott staff 6 Oct 27 11:27 file1.txt -rw-r--r-- 1 scott staff 6 Oct 27 11:29 file3.txt -rw-r--r-- 1 scott staff 6 Oct 27 11:27 file4.txt
As we can see from executing our code above, “file1.txt” was copied to “file4.txt”. However, you may have noticed the creation date was preserved on the new file, unlike with
shutil.copy, which copied “file1.txt” to “file3.txt” and gave it a new creation date.
This method copies the content of a source file into a destination file, from the current source-file position. What this means is that if you read data from your source file object, then the position you stop reading at is the position
copyfileobj starts copying from.
The syntax is as follows:
shutil.copyfileobj(src_file_object, dest_file_object[, length])
The meanings of source and destination file parameters are similar to the previous commands, but now, they refer to objects. The length parameter is optional and represents the buffer size that is the number of bites kept in memory during the copy process. This option can be useful when copying very large files, as it can speed up the copying process and avoids uncontrolled memory usage.
For example the following code will copy “file1.txt” into “file5.txt”
import shutil filename1 = 'file1.txt' fileA = open(filename1, 'rb') filename2 = 'file5.txt' fileB = open(filename2, 'wb') shutil.copyfileobj(fileA, fileB)
As we can see, in order to use
copyfileobj, we need to open the files in binary mode (which is the “b” part of “rb” and “wb”). In addition, the source file must be opened as readable and the destination file must be opened as writable (the “r” and “w” parts, respectively).
Copying Files with the os Module
The os module provides a way to use the operating system functionality to copy your files. In most (if not all) of the examples from here on out we provide examples that work for both Windows and Unix. The examples are different because of the shell commands used, so be sure to pay attention to how each function call is labeled in the Python comments.
This method opens a pipe to or from your command. However, note that this method was deprecated in Python 2.6, so we don’t recommend using it unless you have to. As an alternative, the Python documentation advises us to use methods from the subprocess module instead.
The syntax is as follows:
os.popen(cmd[, mode[, bufsize]])
Here the value returned is a file object that is connected to the pipe. This object can be read from or written to depending on the mode. The default mode is ‘r’, which allows reading of the file contents.
The example below will copy “file1.txt” into “file6.txt”:
import os /* Windows*/ os.popen('copy file1.txt file6.txt') /* Unix */ os.popen('cp file1.txt file6.txt')
Running the command in this way is exactly the same as if you ran it directly from the command line of your terminal.
This method executes the specified command in a subshell. It is available for both Unix and Windows. The syntax is as follows:
command is a string containing the DOS or Unix shell command. In our case, this is where we’ll put the
For example, the following code will copy “file1.txt” into “file7.txt”
import os /* Windows */ os.system('copy file1.txt file7.txt') /* Unix */ os.system('cp file1.txt file7.txt')
This looks identical to the previous
os.popen command we just used, but the command is executed in a subshell, which means it is executed in a separate thread in parallel to your executing code. To wait for its completion, you need to call
.wait() on the object returned by
Copying Files with the subprocess Module
The subprocess module intends to replace some methods in the
os module (particularly
os.system and the
os.spawn* methods), and it presents two main methods to access the operating system commands. These methods are
check_output. Once again, for Unix systems, the command “copy file1.txt file2.txt” should be replaced by “cp file1.txt file2.txt”.
The Python documentation recommends us to use the
call method to launch a command from the operating system.
The syntax is as follows:
subprocess.call(args, *, stdin=None, stdout=None, stderr=None, shell=False)
args parameter will include our shell command. However, a word of caution, as the Python documentation warns us that using
shell=True can be a security risk.
Using this function call, we can run our copy command as follows:
import subprocess /* Windows */ status = subprocess.call('copy file1.txt file8.txt', shell=True) /* Unix */ status = subprocess.call('cp file1.txt file8.txt', shell=True)
As the example above shows, we simply need to pass a string with the shell command, as before.
And as expected, the operating system will copy “file1.txt” into a file named “file8.txt”.
This method also allows us to execute a command within a shell. It is very much like the
subprocess.run command, except that by default it pipes data from stdout as encoded bytes. The syntax is as follows:
subprocess.check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False)
args parameter includes the shell command we want to use. Once again, the Python documentation warns us of using
shell=True, so use this method with caution.
In the following code we’ll copy “file1.txt” to “file9.txt” using the
import subprocess /* Windows */ status = subprocess.check_output('copy file1.txt file9.txt', shell=True) /* Unix */ status = subprocess.check_output('cp file1.txt file9.txt', shell=True)
And as with all of the commands we’ve shown in this article, this will copy the file “file1.txt” to the destination we specified, which is “file9.txt” here.
Python Example for Beginners
Two Machine Learning Fields
There are two sides to machine learning:
- Practical Machine Learning:This is about querying databases, cleaning data, writing scripts to transform data and gluing algorithm and libraries together and writing custom code to squeeze reliable answers from data to satisfy difficult and ill defined questions. It’s the mess of reality.
- Theoretical Machine Learning: This is about math and abstraction and idealized scenarios and limits and beauty and informing what is possible. It is a whole lot neater and cleaner and removed from the mess of reality.
Data Science Resources: Data Science Recipes and Applied Machine Learning Recipes
Introduction to Applied Machine Learning & Data Science for Beginners, Business Analysts, Students, Researchers and Freelancers with Python & R Codes @ Western Australian Center for Applied Machine Learning & Data Science (WACAMLDS) !!!
Latest end-to-end Learn by Coding Recipes in Project-Based Learning:
Disclaimer: The information and code presented within this recipe/tutorial is only for educational and coaching purposes for beginners and developers. Anyone can practice and apply the recipe/tutorial presented here, but the reader is taking full responsibility for his/her actions. The author (content curator) of this recipe (code / program) has made every effort to ensure the accuracy of the information was correct at time of publication. The author (content curator) does not assume and hereby disclaims any liability to any party for any loss, damage, or disruption caused by errors or omissions, whether such errors or omissions result from accident, negligence, or any other cause. The information presented here could also be found in public knowledge domains.