PostgreSQL Example – How to Delete Duplicates

Delete Duplicates

Create Table Of Elves

- Create table called elves
CREATE TABLE elves (
    - string variable
    name varchar(255),
    - integer variable
    age int,
    - string variable
    race varchar(255),
    - string variable
    alive varchar(255)
)

Insert Rows Into Elf Table


INSERT INTO elves (name, age, race, alive)
VALUES ('Dallar Woodfoot', 25, 'Elf', 'Yes'),
       ('Cordin Garner', 29, 'Elf', 'Yes'),
       ('Keat Knigh', 24, 'Elf', 'Yes'),
       ('Keat Knigh', 24, 'Elf', 'Yes'),
       ('Keat Knigh', 24, 'Elf', 'Yes'),
       ('Keat Knigh', 24, 'Elf', 'Yes'),
       ('Colbat Nalor', 124, 'Elf', 'Yes')

View Elves Table

- Retrieve all rows from the view Elf
SELECT * FROM elves
name age race alive
Dallar Woodfoot 25 Elf Yes
Cordin Garner 29 Elf Yes
Keat Knigh 24 Elf Yes
Keat Knigh 24 Elf Yes
Keat Knigh 24 Elf Yes
Keat Knigh 24 Elf Yes
Colbat Nalor 124 Elf Yes

Drop Duplicates

Note: Normally we would use a unique identify column (e.g. person ID, product ID, etc.). However, since we don’t have a unique ID column we can use PostgreSQL’s internal system column, ctid. Full documentation on ctid and other system columns in avaliable here.


- Delete from the elves, calling it copy1
DELETE FROM elves copy1
- Using a second copy of elves, called copy2
USING elves copy2
- Where the internal PostgreSQL system column, ctid is smaller
WHERE copy1.ctid < copy2.ctid
  - And all other columns are the same
  AND copy1.name = copy2.name
  AND copy1.age = copy2.age
  AND copy1.race = copy2.race
  AND copy1.alive = copy2.alive
name age race alive
Dallar Woodfoot 25 Elf Yes
Cordin Garner 29 Elf Yes
Keat Knigh 24 Elf Yes
Colbat Nalor 124 Elf Yes

 

Python Example for Beginners

Two Machine Learning Fields

There are two sides to machine learning:

  • Practical Machine Learning:This is about querying databases, cleaning data, writing scripts to transform data and gluing algorithm and libraries together and writing custom code to squeeze reliable answers from data to satisfy difficult and ill defined questions. It’s the mess of reality.
  • Theoretical Machine Learning: This is about math and abstraction and idealized scenarios and limits and beauty and informing what is possible. It is a whole lot neater and cleaner and removed from the mess of reality.

Data Science Resources: Data Science Recipes and Applied Machine Learning Recipes

Introduction to Applied Machine Learning & Data Science for Beginners, Business Analysts, Students, Researchers and Freelancers with Python & R Codes @ Western Australian Center for Applied Machine Learning & Data Science (WACAMLDS) !!!

Latest end-to-end Learn by Coding Recipes in Project-Based Learning:

Applied Statistics with R for Beginners and Business Professionals

Data Science and Machine Learning Projects in Python: Tabular Data Analytics

Data Science and Machine Learning Projects in R: Tabular Data Analytics

Python Machine Learning & Data Science Recipes: Learn by Coding

R Machine Learning & Data Science Recipes: Learn by Coding

Comparing Different Machine Learning Algorithms in Python for Classification (FREE)

Disclaimer: The information and code presented within this recipe/tutorial is only for educational and coaching purposes for beginners and developers. Anyone can practice and apply the recipe/tutorial presented here, but the reader is taking full responsibility for his/her actions. The author (content curator) of this recipe (code / program) has made every effort to ensure the accuracy of the information was correct at time of publication. The author (content curator) does not assume and hereby disclaims any liability to any party for any loss, damage, or disruption caused by errors or omissions, whether such errors or omissions result from accident, negligence, or any other cause. The information presented here could also be found in public knowledge domains.  

Google –> SETScholars