How to fix Duplicate sources.list entry?

Question

I keep getting this warning whenever I try to run sudo apt-get update.

W: Duplicate sources.list entry http://archive.ubuntu.com/ubuntu/ precise-updates/main i386 Packages (/var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_precise-updates_main_binary-i386_Packages)
W: You may want to run apt-get update to correct these problems

Below is the output from /etc/apt/sources.list file:

deb http://archive.ubuntu.com/ubuntu precise main restricted
deb-src http://archive.ubuntu.com/ubuntu precise main restricted

deb http://archive.ubuntu.com/ubuntu precise-updates main restricted
deb-src http://archive.ubuntu.com/ubuntu precise-updates main restricted

deb http://archive.ubuntu.com/ubuntu precise universe
deb-src http://archive.ubuntu.com/ubuntu precise universe
deb http://archive.ubuntu.com/ubuntu precise-updates universe
deb-src http://archive.ubuntu.com/ubuntu precise-updates universe

deb http://archive.ubuntu.com/ubuntu precise multiverse
deb-src http://archive.ubuntu.com/ubuntu precise multiverse
deb http://archive.ubuntu.com/ubuntu precise-updates multiverse
deb-src http://archive.ubuntu.com/ubuntu precise-updates multiverse

deb http://archive.ubuntu.com/ubuntu precise-security main restricted
deb-src http://archive.ubuntu.com/ubuntu precise-security main restricted
deb http://archive.ubuntu.com/ubuntu precise-security universe
deb-src http://archive.ubuntu.com/ubuntu precise-security universe
deb http://archive.ubuntu.com/ubuntu precise-security multiverse
deb-src http://archive.ubuntu.com/ubuntu precise-security multiverse

How do I fix it?

Dan · Answer 1 · 2015-05-25 09:06:37Z

Your sources.list really has duplicate entries.
First Let's see the correct form of a repository line or source line:
The correct format of repository source line is

<type of repository>  <location>  <dist-name> <components>

For example, a repo line can be like this one

  deb http://archive.ubuntu.com/ubuntu precise main

^{Here, it means, the repository is for binary packages, which are hosted in http://archive.ubuntu.com/ubuntu and this repository is for Ubuntu precise (12.04) and this repository contains the main (software which are officially supported by Canonical) component.}

Type: The type can be deb and deb-src. deb means a binary repository where deb-src means a source repository
Location: http://archive.ubuntu.com/ubuntu location of the repository.
Dist-name: The distribution name of Ubuntu release. For Ubuntu 12.04 it is precise, for 11.10 it is oneiric.
^{You can visit the Ubuntu Wiki to view an updated list of Ubuntu releases and their code names.}
Component: It can be main, universe, multiverse and restricted. These words indicates the level of supports for the packages and the licensing status.

See this page for more information.
Please take note that, you can add one or more component in a line, so "main", "universe", "restricted" and "multiverse" can be in a single line. Also note, Though you add more than one component in a single line, APT system considers them as separate line containing only one component.
So, If your sources.list have a line like this

deb http://archive.ubuntu.com/ubuntu precise universe

Then it can't have another line like the below (which your files have)

deb http://archive.ubuntu.com/ubuntu precise main universe

which is equivalent of these two lines

deb http://archive.ubuntu.com/ubuntu precise main
deb http://archive.ubuntu.com/ubuntu precise universe

Because, you are duplicating universe twice, so there will be an error for that duplicate. An error is given for each duplicate found and your file has 6 duplicates there.

Solution:

After analyzing your sources.list file, I found that, It is a basic one which is supplied by Ubuntu by default. You can build a default file following the procedure:

Open a terminal and first rename the existing sources.list file to sources.list.bak file. (we can safely remove that, but caution is good).
```
 sudo mv /etc/apt/sources.list /etc/apt/sources.list.bak
```
Then open software-properties-gtk to build a new one. Type the below command in a terminal or in dash command prompt with pressing Alt+F2
```
software-properties-gtk
```
^{Note, for Ubuntu version 11.10 or earlier you have to use gksu before the software-properties-gtk command}
After the window is open,
- Select all four categories in Ubuntu Software tab
- Select precise-security, precise-updates and precise-backports category in Updates tab.
- Select Canonical partner and Independent category from Other software tab.

That's it. You have now a default sources.list file without error. Compare this file with the previous one, if you want.

Update to deal with `sources.list.d` dir's files

Sometimes a duplicate entry can be in a file in the /etc/apt/sources.list.d directory, which is also used by APT. So, you should look at that directory and see if there is any duplicate in those files. It is not necessary for duplicates being in the same file.

Example case:

A user had this error message showing while doing sudo apt-get update.

W: Duplicate sources.list entry http://dl.google.com/linux/chrome/deb/ stable/main amd64 Packages (/var/lib/apt/lists/dl.google.com_linux_chrome_deb_dists_stable_main_binary-amd64_Packages)
W: Duplicate sources.list entry http://dl.google.com/linux/chrome/deb/ stable/main i386 Packages (/var/lib/apt/lists/dl.google.com_linux_chrome_deb_dists_stable_main_binary-i386_Packages)
W: You may want to run apt-get update to correct these problems

But there was no entry with http://dl.google.com in the main /etc/apt/sources.list file. Looking at the /etc/apt/sources.list.d/ directory we found these files:
output of ls /etc/apt/sources.list.d/:

aims-sagemath-precise.list
aims-sagemath-precise.list.save
dropbox.list
dropbox.list.save
fossfreedom-packagefixes-precise.list
fossfreedom-packagefixes-precise.list.save
google-chrome.list
google-chrome.list.save
google.list
google.list.save
sagemath-monolithic-precise.list
sagemath-monolithic-precise.list.save

The files google-chrome.list and google.list looked promising for containing duplicates. So, investing the content of both files were necessary
Output of cat /etc/apt/sources.list.d/google-chrome.list:

deb http://dl.google.com/linux/chrome/deb/ stable main

and of cat /etc/apt/sources.list.d/google.list

deb http://dl.google.com/linux/chrome/deb/ stable main

So, there was a duplicate entries in the sources of Apt. Since google-chrome.list only contain a single apt line and it was also listed in google.list file we can safely remove this file with the command

sudo rm /etc/apt/sources.list.d/google-chrome.list

Then the problem was solved

+1 This is a great answer. Fixed the problem and fully explained what why it had occurred. — akmad, Apr 27 '13 at 9:13
What did I do to cause myself to have duplicate entries in my sources.list.d? — Seanny123, Nov 11 '13 at 9:19
link to Solve Duplicate source.list entry mentioned here solved my confusion. Try, and if it helps you, praise O.A.K. — Peter Masiar, Jul 21 '14 at 17:57

Ron · Answer 2 · 2015-08-17 11:42:45Z

up vote 51 down vote

+50

For Ubuntu 12.04
Graphically:
Press Alt + F2 and paste software-properties-gtk (or you can open "Software Center" then go to "Edit" > "Software Sources"). Go to tab "Other Software", choose the duplicate entry and press the "Remove" button.
enter image description here

If you need a command line option, here it is:

cat /etc/apt/sources.list | perl -ne '$H{$_}++ or print' > /tmp/sources.list && sudo mv /tmp/sources.list /etc/apt/sources.list

The command does this:
cat reads the file and passes the content to perl which removes the duplicate lines. The result is then saved > in a temporary file which is then moved to replace the original /etc/apt/sources.list file.

edited Aug 17 at 11:42

Ron
5,72931230

answered May 4 '12 at 22:27

desgua
19.6k45795

Your script worked perfectly. – Lucio Dec 20 '13 at 21:01

1

Just how to fix it, without confusion and all the too deep details. Thank you! – Peter Masiar Jul 21 '14 at 18:00

It actually worked! Could not, for some reason, install git on a 12.04 either. Instead of remove, I just checked the apparent duplicated "Canonical partners". – davidkonrad Oct 15 '14 at 12:10

add a comment |

Qasim · Answer 3 · 2013-05-08 05:49:46Z

How bad are duplicate entries in sources.list?

I don’t know how bad it is, but i don’t like sudo apt-get update showing me duplicate entries.
By the way its not that bad, its just showing you that you have duplicate entries.

SourcesList

The sources.list file is a key factor in adding or upgrading applications to your Ubuntu installation. This is also used by your system for system updates. The file is basically the roadmap for your system to know where it may download programs for installation or upgrade.
Its just like Windows update

You can remove duplicate entries in few easy steps with Y PPA Manager

 sudo add-apt-repository ppa:webupd8team/y-ppa-manager -y
 sudo apt-get update
 sudo apt-get install y-ppa-manager -y

Open y-ppa-manager form Dash

Enter You Admin Password

Double Click On Advanced

Scan & Remove Duplicate PPA's & Click Ok

It will take some time ( 1 or 2 Mints ) To Scan & Remove Duplicate PPA's

In my case, Y PPA kept saying no duplicates found. I made a backup of the source.list removed it, and ran sudo apt-get update to recreate the source list (ubuntu 12.04). This worked for me. — michel.iamit, Jul 5 '13 at 8:27

Eric Carvalho · Answer 4 · 2012-09-01 19:20:10Z

up vote 15 down vote

The partner repository is duplicated inside /etc/apt/sources.list and /etc/apt/sources.list.d/precise-partner.list.
Just remove those "precise-partner" files as the partner repository is already present in sources.list.

sudo rm /etc/apt/sources.list.d/precise-partner.*

answered Sep 1 '12 at 19:20

Eric Carvalho
21.8k135185

I'm getting rm: cannot remove '/etc/apt/sources.list.d/precise-partner.*': No such file or directory – Dennis Mar 30 '13 at 23:35

1

Try sudo rm /etc/apt/sources.list.d/*partner*. Also check your /etc/apt/sources.list, look for duplicate lines. – Eric Carvalho Mar 31 '13 at 0:51

No luck, I don't have any files with "partner" in the name in that directory. And /etc/apt/sources.list doesn't contain any duplicates according to uniq. – Dennis Jun 23 '13 at 23:03

add a comment |

wojox · Answer 5 · 2012-07-04 05:16:21Z

up vote 10 down vote

Make a back up copy of your sources.list:

sudo cp /etc/apt/sources.list{,.backup}

Now remove the original and update:

sudo rm /etc/apt/sources.list && sudo apt-get update

answered Jul 4 '12 at 5:16

wojox
5,1662350

How can he update without sources.list file? – Anwar Shah Jul 4 '12 at 5:27

It will repopulate. =) – wojox Jul 4 '12 at 5:28

@wojox It works, thanks. – quantme Dec 16 '12 at 5:26

mine did not repopulate – conman253 Feb 14 '14 at 22:38

Revert to the one you backed up – wojox Feb 15 '14 at 13:42

add a comment |

Amjed · Answer 6 · 2012-04-11 05:11:33Z

I had the same problem, open software sources. unchecked "Canonical Partners". The Conflict is between "Canonical Partners" and " Canonical partners Added by software Center"

Jorge Castro · Answer 7 · 2012-10-05 00:05:05Z

up vote 7 down vote

If you couldn't figure it out in your source.list, then do this:

Left-click on settings icon (at the extreme top corner of your screen) and select System Settings.
Click on Software Sources and move to Other Software.
Uncheck one of the 'Canonical Partners(Source Code) -software packaged by Canonical for their partners' files and one of the 'Canonical Partners -software packaged by Canonical for their partners' files also and click Close to leave.
Simultaneously press CTRL+ALT+T keys to open terminal and type sudo apt-get update.

edited Oct 5 '12 at 0:05

Jorge Castro
25.3k83362571

answered Apr 11 '12 at 6:15

all4naija
1,2411720

I followed your suggestions, but still I am getting this error. – HSS Apr 11 '12 at 8:08

Please, post the image of your 'Software Sources' here to know the culprit. – all4naija Apr 11 '12 at 18:19

Please, post the image of your 'Other Software' here. – all4naija Apr 11 '12 at 18:22

add a comment |

rearlight · Answer 8 · 2012-05-07 06:07:05Z

up vote 5 down vote

For Ubuntu 9.10 - 12.04
There is an app called Y-PPA-Manager which can do that and much more.
You can install it like that from a Terminal:

sudo add-apt-repository ppa:webupd8team/y-ppa-manager
sudo apt-get upgrade
sudo apt-get install y-ppa-manager

After that, just start the application an go to the Advanced menu.
main window

Then select "Scan and remove duplicate PPAs" and hit OK.
advanced window

edited May 7 '12 at 6:07

answered May 6 '12 at 13:25

rearlight
538816

It's cool tool but it doesn't fix problem. I did this choice and after this have the same warnings. – nazar_art Jul 25 '13 at 10:28

add a comment |

DeaDSouL · Answer 9 · 2011-03-17 01:50:13Z

I googled for such a tool, but I didn't find any.. So, I ended up coding one myself with PHP.
Sorry for the dirty unorganized source code..

chkdup - Screenshot:

Author

Mubarak Alrashidi (DeaDSouL)

Requirements:

PHP.
Sudo.

Installation:

Open the terminal.
To install PHP if you don't have it, type: sudo apt-get install php5 php5-cli.
Download the chkdup.
Extract the file anywhere you like.
Type: sudo sh install.sh.

Uninstall:

Open the terminal.
Go to the path you extracted the tool in.
Type: sudo sh uninstall.sh.

How to use it:

Just open the terminal and type sudo chkdup.

FAQs:

What does the install.sh do ?

The installer will move the chkdup.php to /opt/ then will create a symbolic link in /usr/bin/.

What about the uninstall.sh ?

The uninstaller will remove the following files:
1. install.sh
2. /opt/chkdup.php
3. /usr/bin/chkdup
4. itself "uninstall.sh"

Is it safe to use chkdup ?

Yes it is, since it takes a backup of the current sources.list before attempting to do anything. But I'm not responsible if anything happened. The full responsibility will be yours.

License

GNU General Public License
And goodluck

Thank you very much for the code. – Vassilis Mar 17 '11 at 8:27 — Vassilis, Mar 17 '11 at 8:27
just tried it out, nice and simple :-) nice :-) – bmbaker Jan 28 '12 at 19:40 — bmbaker, Jan 28 '12 at 19:40

Jeff Welling · Answer 10 · 2012-04-10 15:01:19Z

It sounds like you have two identical lines in your /etc/apt/sources.list file.
Go to terminal, and enter sudo editor /etc/apt/sources.list, then look for any identical lines in that file, and if you find them, delete them. After your done, save and exit, and then do sudo apt-get update and that should resolve your problem.
If that doesn't solve your problem then you're going to have to use the trail and error approach. First, create a backup of your sources.list file, then in the actual file go through and comment out or delete each line, saving and apt-get update-ing after each line has been commented out or deleted. Going through this file line by line commenting out different lines will eventually tell you which line is the superfluous one. Don't forget to uncomment any lines you commented if apt-get update still shows the same prompt, otherwise you'll be left with no sources in your sources.list file.

Jorge Castro · Answer 11 · 2012-05-04 19:14:48Z

10.10 and Earlier

Applications > Ubuntu Software Center > Edit > Software Sources > Other Software**
OR
System > Administration > Synaptic Packet Manager > Settings > Repositories**
alt text

Select and Remove duplicate entries.

Eliah Kagan · Answer 12 · 2012-07-07 13:11:16Z

up vote 2 down vote

Did you check ls /etc/apt/sources.list.d/? Maybe you have some "extra" repositories in there?

edited Jul 7 '12 at 13:11

Eliah Kagan
47.8k12133230

answered Apr 11 '12 at 11:41

konrad
824414

add a comment |

Community · Answer 13 · 2013-11-11 04:10:08Z

up vote 2 down vote

Install ubuntu-tweak and remove the duplicate entries from there,it will be easy
for you..

sudo add-apt-repository ppa:tualatrix/ppa
sudo apt-get update
sudo apt-get install ubuntu-tweak

edited Nov 11 '13 at 4:10

Community♦
1

answered Nov 18 '10 at 15:43

karthick87
20.4k31125188

This does not say how to do it. – RolandiXor♦ Nov 15 '14 at 17:43

add a comment |

Pitto · Answer 14 · 2010-11-18 17:12:43Z

I try to complete the possibilities offered from other users using terminal, since that's what you asked, mainly:
login if you're using a text based install or
Press alt+f2 and digit gnome-terminal if you're using GUI
once there digit
sudo nano /etc/apt/sources.list
(To understand better: in the folder /etc you can find (nearly) all configuration files of your computer and in the apt folder you find the souces.list file that contains all the repositories that apt-get uses to update or install and upgrade software)
edit visually the file removing or commenting with # the duplicated rows.
Press control + x to exit and choose y to save the file n to quit without saving.
Done that check your work with an update:
sudo apt-get update
Have fun!

I also think that evolved text editor like vim can automatically find duplicates but I don't know how to do that any kind of info would be interesting — Pitto, Nov 18 '10 at 17:17

Eliah Kagan · Answer 15 · 2012-06-15 08:33:55Z

up vote 1 down vote

In my case the duplicates were in the prerequists-sources.list file. I was able to continue the upgrade by running the following commands in the Terminal:

sudo mv /etc/apt/sources.list.d/prerequists-sources.list /tmp
sudo apt-get update

Then go to the Update Manager and install the updates.

edited Jun 15 '12 at 8:33

Eliah Kagan
47.8k12133230

answered Jun 15 '12 at 8:18

Mathijs Sterk
111

Moving to /tmp is not necessary. Renaming to something which doesn't end with .list is enough – Daniel Alder Oct 13 '14 at 20:03

add a comment |

fabricator4 · Answer 16 · 2012-12-14 18:59:51Z

up vote 1 down vote

Just sudo gedit sources.list of the file where the Duplicate is and remove the line or put the # before the line!

edited Dec 14 '12 at 18:59

fabricator4
6,13311734

answered Dec 14 '12 at 18:41

Micha
111

add a comment |

slhck · Answer 17 · 2013-01-05 16:12:07Z

Use the Ubuntu Sources List Generator.

Run sudo nautilus in terminal
/etc/apt/sources.list
Right click and open the file with a text editor (if you click on it, it will open the software sources GUI)
Simply copy and paste into the sources.list and save.
Exit terminal and you're done.

Google seems to have a problem, so I'd suggest leaving that out. I have done this on Ubuntu 12.10 (Studio version with Ubuntu desktop added and Unity for another desktop sign in).

Seann · Answer 18 · 2013-04-25 19:36:15Z

Well, I solved the Duplicate Entry problem myself by renaming /etc/apt/sources.d (as root) to something else and running sudo apt-get update to re-establish the directory and it's contents.

Eric Carvalho · Answer 19 · 2013-06-24 00:43:50Z

up vote 1 down vote

In /etc/apt/sources.list you must comment (#) duplicate repository and save changes. After that repeat apt-get update.

edited Jun 24 '13 at 0:43

Eric Carvalho
21.8k135185

answered May 28 '13 at 9:07

Snorri
111

add a comment |

Braiam · Answer 20 · 2013-09-03 13:12:55Z

up vote 1 down vote

Went through /etc/apt/sources.list manually sudo nano /etc/apt/sources.list and removed the duplicates. After apt-get update, the problem was solved.

edited Sep 3 '13 at 13:12

Braiam
31.8k1567128

answered Sep 3 '13 at 12:01

Anderverhaal
132

add a comment |

Smarteralec · Answer 21 · 2012-05-07 22:11:42Z

No, there isn't a single command line to find and delete duplicated entries in the /etc/apt/sources.list file.
This is because the suggested command lines, which merely duplicate the function of the much simpler built-in uniq command, will only work on a SORTED file and will only remove ADJACENT lines. Furthermore, those commands will only remove lines consisting of an identical string of characters.
Duplicate entries reported by apt-get update will consist of duplicate function entries, such as a repository being included in both its i386 and amd64 variants. The easy and effective way to remove these is to note which repositories are reported as duplicates by apt-get update and remove them via the Software Center. Open it and choose Edit -> Software Sources -> Other Software tab. Simply look for the duplicate entries and uncheck them. (This is also an opportunity to remove any source-code repositories if you're not compiling the packages).
However, it should be noted that apt-get update doesn't only find duplicate entries via sources.list, but includes repository files located in /var/lib/apt/lists/. Deleting duplicate entries from there will only temporarily remove the apt-get update error messages if Ubuntu thinks it needs them, as the files will be automatically re-installed. The fact that apt-get update will report some repositories as duplicates, and then suggest that that you run apt-get update itself to repair them, is a clue that you really don't need to worry about that error message.

Nathan Osman · Answer 22 · 2012-07-04 04:54:22Z

up vote 0 down vote

S="/etc/apt/sources.list" ;
S2="$S ${S}.d/*.list" ;
grep -b "^deb\`cat $S2 |
    grep -i "^deb[[:space:]]http" |
    sort |
    uniq -dc |
    sed -e 's;[[:space:]]\\+[[:digit:]]\\+[[:space:]]\\+deb\\(.\\+$\\);\\1;g'\`$" $S2

* Line breaks inserted for readability.

edited Jul 4 '12 at 4:54

Nathan Osman
15.6k1894181

answered Jul 3 '12 at 23:59

Tony Saratonev
112

2

Please explain the answer or risk having it removed. – jokerdino♦ Jul 4 '12 at 4:55

Welcome to Ask Ubuntu! As jokerdino as indicated, this answer would benefit substantially, if you were to edit it to add some explanation (or at least to tell the user exactly what to do with these lines). – Eliah Kagan Jul 4 '12 at 10:34

This answer is wrong because it may does remove duplicate entries but it does not remove duplicate source entries. see here for an explanation – Anwar Shah Sep 6 '12 at 5:26

add a comment |

Stormvirux · Answer 23 · 2013-09-14 10:46:40Z

up vote 0 down vote

Execute the following to remove the duplicates

cat /etc/apt/sources.list | perl -ne '$H{$_}++ or print' > /tmp/sources.list && sudo mv /tmp/sources.list /etc/apt/sources.list

answered Sep 14 '13 at 10:46

Stormvirux
2,7551323

more info on this please – cipricus Nov 5 '13 at 12:32

add a comment |

Stevie G · Answer 24 · 2015-05-25 08:38:21Z

In the case of

W: Duplicate sources.list entry http://dl.google.com/linux/chrome/deb/ stable/main amd64 Packages (/var/lib/apt/lists/dl.google.com_linux_chrome_deb_dists_stable_main_binary-amd64_Packages)

Open: sources.list.d
rm google.list
so only google-chrome.list remains

asked	3 years ago
viewed	174814 times
active	2 months ago

blog kopas

Rabu, 04 November 2015

current community

your communities

more stack exchange communities

How to fix Duplicate sources.list entry?

24 Answers 24

Solution:

Update to deal with `sources.list.d` dir's files

Example case:

How bad are duplicate entries in sources.list?

SourcesList

chkdup - Screenshot:

Author

Requirements:

Installation:

Uninstall:

How to use it:

FAQs:

License

10.10 and Earlier

protected by Community♦ Jan 7 '14 at 12:19

Not the answer you're looking for? Browse other questions tagged apt configuration software-sources or ask your own question.

Linked

Hot Network Questions

Share

0 komentar:

Posting Komentar

Popular Posts

Arsip Blog

Mengenai Saya

About

Introductory Video

blog kopas

Rabu, 04 November 2015

current community

your communities

more stack exchange communities

How to fix Duplicate sources.list entry?

24 Answers 24

Solution:

Update to deal with sources.list.d dir's files

Example case:

How bad are duplicate entries in sources.list?

SourcesList

chkdup - Screenshot:

Author

Requirements:

Installation:

Uninstall:

How to use it:

FAQs:

License

10.10 and Earlier

protected by Community♦ Jan 7 '14 at 12:19

Not the answer you're looking for? Browse other questions tagged apt configuration software-sources or ask your own question.

Linked

Related

Hot Network Questions

Share

0 komentar:

Posting Komentar

Popular Posts

Arsip Blog

Mengenai Saya

About

Introductory Video

Update to deal with `sources.list.d` dir's files