Reshaping Data with R
Introduction
In predictive modeling, it is often necessary to reshape the data to make it ready for conducting analysis or building models. The process of transforming the data into a clear, simple, and desirable form is an integral component of data science. The most common reshaping process is converting the data from wide to long format and vice versa.
In this guide, you will learn about techniques for reshaping data in R. There are many powerful libraries in R to perform this task, and you will learn about two such packages: reshape2 and tidyr.
Data
In this guide, we will use simple fictitious data of grades (out of 100) for five students in three subjects. Let’s start by creating sample data using the lines of code below. We will also convert the roll number to a factor variable, as this will help in data wrangling.
df_wide <- read.table(header=TRUE, text=' Rollno name math science english 1 Name1 80 72 77 2 Name2 65 80 71 3 Name3 45 54 67 4 Name4 95 73 70 ') df_wide$Rollno <- factor(df_wide$Rollno) head(df_wide)
Output:
| | Rollno | name | math | science | english |
|--- |-------- |------- |------ |--------- |--------- |
| 1 | 1 | Name1 | 80 | 72 | 77 |
| 2 | 2 | Name2 | 65 | 80 | 71 |
| 3 | 3 | Name3 | 45 | 54 | 67 |
| 4 | 4 | Name4 | 95 | 73 | 70 |We’ll use this simple data to perform the reshaping tasks in the subsequent sections.
Using ‘reshape2’
reshape2 is an R package written by Hadley Wickham that makes it easy to transform data between the wide and the long formats. Wide data has a column for every variable, while this is not compulsory in long data.
Wide to Long Conversion
Sometimes you may be required to reshape wide data into a long format. The melt function in ‘reshape2’ performs this task.
The first line of code below loads the library, while the second line melts the data into the long format. The arguments to be specified are outlined below:
-
Argument ‘id.vars’: Specifies the variables to be retained and not split apart
-
Argument ‘measure.vars’: Represents the source columns
-
Argument ‘variable.name’: Represents the destination column that will identify the original variables from which the values came
-
Argument ‘value.name’: Represents the values, which in our data are the exam marks
The resultant data is stored in a new data frame, ‘df_long’. The third line of code below prints this resulting data set, which now contains 12 observations of 4 variables.
library(reshape2) df_long <- melt(df_wide, id.vars=c("Rollno", "name"), measure.vars=c("math", "science", "english" ), variable.name="subject", value.name="marks" ) df_long
Output:
| Rollno | name | subject | marks |
|--------- |------- |--------- |------- |
| 1 | Name1 | math | 80 |
| 2 | Name2 | math | 65 |
| 3 | Name3 | math | 45 |
| 4 | Name4 | math | 95 |
| 1 | Name1 | science | 72 |
| 2 | Name2 | science | 80 |
| 3 | Name3 | science | 54 |
| 4 | Name4 | science | 73 |
| 1 | Name1 | english | 77 |
| 2 | Name2 | english | 71 |
| 3 | Name3 | english | 67 |
| 4 | Name4 | english | 70 |Long to Wide Conversion
In the previous section, we created long data from wide data. To convert long data back into a wide format, we can use the cast function. There are many cast functions, but we will use the dcast function because it is used for data frames.
The lines of code below will perform this conversion.
data_wide <- dcast(df_long, Rollno + name ~ subject, value.var="marks") data_wide
Output:
| | Rollno | name | math | science | english |
|--- |-------- |------- |------ |--------- |--------- |
| 1 | 1 | Name1 | 80 | 72 | 77 |
| 2 | 2 | Name2 | 65 | 80 | 71 |
| 3 | 3 | Name3 | 45 | 54 | 67 |
| 4 | 4 | Name4 | 95 | 73 | 70 | Using ‘tidyr’
The tidyr package can also be used to reshape data. It uses the gather function to convert data from wide to long format and uses the spread function to convert it from long to wide format.
Wide to Long Conversion
The first line of code below loads the library, while the second line reshapes the data to long format. The third line prints the resultant data.
library(tidyr) df_long_tidyr <- gather(df_wide, subject, marks, math:english, factor_key=TRUE) df_long_tidyr
Output:
| Rollno | name | subject | marks |
|--------- |------- |--------- |------- |
| 1 | Name1 | math | 80 |
| 2 | Name2 | math | 65 |
| 3 | Name3 | math | 45 |
| 4 | Name4 | math | 95 |
| 1 | Name1 | science | 72 |
| 2 | Name2 | science | 80 |
| 3 | Name3 | science | 54 |
| 4 | Name4 | science | 73 |
| 1 | Name1 | english | 77 |
| 2 | Name2 | english | 71 |
| 3 | Name3 | english | 67 |
| 4 | Name4 | english | 70 |Long to Wide Conversion
The lines of code below return the original data, which was in wide format, by using the ‘spread’ function.
df_wide_tidyr <- spread(df_long_tidyr, subject, marks) df_wide_tidyr
Output:
| | Rollno | name | math | science | english |
|--- |-------- |------- |------ |--------- |--------- |
| 1 | 1 | Name1 | 80 | 72 | 77 |
| 2 | 2 | Name2 | 65 | 80 | 71 |
| 3 | 3 | Name3 | 45 | 54 | 67 |
| 4 | 4 | Name4 | 95 | 73 | 70 |Transposing the Data
The transpose function reverses rows into columns and vice versa. It is perhaps the simplest method of reshaping a dataset, and it uses the t() function to transpose a matrix or a data frame, as shown below.
t(df_wide)
Output:
[,1] [,2] [,3] [,4] Rollno "1" "2" "3" "4" name "Name1" "Name2" "Name3" "Name4" math "80" "65" "45" "95" science "72" "80" "54" "73" english "77" "71" "67" "70"
Conclusion
In this guide, you have learned two popular data reshaping techniques using the ‘reshape2’ and ‘tidyr’ packages. You also learned about transposing data. These tools will help you convert your data into a format that is easier to analyze.
Python Example for Beginners
Two Machine Learning Fields
There are two sides to machine learning:
- Practical Machine Learning:This is about querying databases, cleaning data, writing scripts to transform data and gluing algorithm and libraries together and writing custom code to squeeze reliable answers from data to satisfy difficult and ill defined questions. It’s the mess of reality.
- Theoretical Machine Learning: This is about math and abstraction and idealized scenarios and limits and beauty and informing what is possible. It is a whole lot neater and cleaner and removed from the mess of reality.
Data Science Resources: Data Science Recipes and Applied Machine Learning Recipes
Introduction to Applied Machine Learning & Data Science for Beginners, Business Analysts, Students, Researchers and Freelancers with Python & R Codes @ Western Australian Center for Applied Machine Learning & Data Science (WACAMLDS) !!!
Latest end-to-end Learn by Coding Recipes in Project-Based Learning:
Applied Statistics with R for Beginners and Business Professionals
Data Science and Machine Learning Projects in Python: Tabular Data Analytics
Data Science and Machine Learning Projects in R: Tabular Data Analytics
Python Machine Learning & Data Science Recipes: Learn by Coding
R Machine Learning & Data Science Recipes: Learn by Coding
Comparing Different Machine Learning Algorithms in Python for Classification (FREE)
Disclaimer: The information and code presented within this recipe/tutorial is only for educational and coaching purposes for beginners and developers. Anyone can practice and apply the recipe/tutorial presented here, but the reader is taking full responsibility for his/her actions. The author (content curator) of this recipe (code / program) has made every effort to ensure the accuracy of the information was correct at time of publication. The author (content curator) does not assume and hereby disclaims any liability to any party for any loss, damage, or disruption caused by errors or omissions, whether such errors or omissions result from accident, negligence, or any other cause. The information presented here could also be found in public knowledge domains.