logo

compteur depuis le 8 mai 2002
logo2


 Some 'lugdunum' servers
Razorback 900000 users
WSL 180000 users
DS-No 1 150000 users
DS-No 2 150000 users
DS-No 3 150000 users
Server List
ed2khistory

 Our server
Accueil
Nouveautés
Techniques de son succès
Caractéristiques
Bloc d'adresses IP Fr
Ajouter dans Edonkey

EMail

 L'ID du donkey
Comment le récupérer
Son attribution
Utilisation
Qui provoque un Low ID ?

 Connection sharing
Win XP
Win 98SE / Win Me
Win 2000
MAC OS/X
infos partages connexions

 Router configurations
Alcatel Pro
Asante FriendlyNet Câble/DSL FR3004
Bewan LanBooster
Cisco utilisant le NAT
Eicon Diva ISDN
NetGear RP114
Netline
Netopia Rxxx
Zyxel Prestige 310
Autres routeurs
infos routeurs

 Gateways
Linux 2.2
Linux 2.4 et Shorewall
NetBSD 1.5.2
infos passerelles

 Links
FAQ (par MichelSup)
jigle
NetFilter
edonkey2000
Overnet
OPEN-files
Foro oficial del edonkey en español
The Donkey Network
Forum Edonkey 2000fr
Forum Walkyrie
BeEdonkey
Jaquettes Divx
cartonne.tk
www.edonkey-divx.com
ed2k.ch

Créé par Interfaces Web

Building an efficient edonkey server on linux/FreeBSD

Stillman translated this page into german : German Translation
A page in traditional chinese is available here : Chinese Translation

You will find in this page usefull information about our edonkey servers.
Dont hesitate to send us a mail if you think this page lacks some information.

The current program is dserver-16.38.p76.gz
md5sum : be974dd164ec2e10b5a6997cd3fbcf79

Mirrors : Mirror1 or Mirror2 or Mirror3 and nbuser.gz or nbuser_s.gz (static version of nbuser for RedHat 8.0 glibc bug)

You will install it : (hint for the newbies : dont type the '$', this is the shell prompt (can be '#' if you are root))
$ wget ftp://ftp.aubi.de/pub/edonkey/dserver-16.38.p76.gz
$ gzip -d dserver-16.38.p76.gz
$ chmod 755 dserver-16.38.p76
$ rm -f dserver
$ ln -s dserver-16.38.p76 dserver

$ wget ftp://ftp.aubi.de/pub/edonkey/nbuser.gz
$ gzip -d nbuser.gz
$ chmod 755 nbuser

$ wget ftp://ftp.aubi.de/pub/edonkey/dlaunch
$ chmod 755 dlaunch

This program is based on the version 16.38 from Jed McCaleb, patched by us to cut the CPU power needed by a fifty factor. It saves also some memory. The patches were done using hand generated assembly language with the results given by the excellent oprofile package, which flagged the hot spots of the original program for us. Some functions were rewritten to take off some stupid loops. Some locks/unlocks were avoided using 'lock' prefix on several instructions.

arrow It detects BOT PRO and antisocial edonkey clients

arrow It corrects numerous bugs.

arrow It can reject LowID clients : connections from firewalled clients, or multiple connections from a same IP address. LowID clients have a BIG impact on the traffic on a server. Since .p56, we can put a limit on the percentage of allowed LOWID users on a server (default 20%)

arrow Original dserver sends every 70 seconds a 'ping' packet to each client. These 'ping' contains the number of clients and files.
Our version sends every 10 seconds a 'ping' to 1/10th of clients. This is to lower the impact of the pings on the Upload, an to let the server know the clients shutdown more uniformly.

arrow It has a filter to limit the number of shares for each client. The first limit (soft limit) is 1000 by default. If a client publishes more than 1000 shares, the shares in excess are ignored by the server. Moreover, if the number of shares exceeds the hard limit (5000 by default) the client is kicked off the server, in order to save the bandwidth.
Studies have demonstrated that having too many shares is a very bad thing for the servers and the edonkey network. Some lazy users are sharing all the files of their computer (We saw a record of 196881 shares on one client !!)

arrow Only one network segment is used by the server when he sends an edonkey packet to a TCP client. The original dserver is lazy and sends two packets. Note that old edonkey client programs have the same bug. We are pleased that Emule 0.20 corrected this problem. The 10.44 version of hybrid edonkey did the correction too.

new version 16.40 in RC4 stage :

  • Tests of alpha version started Friday Sep 5th

  • First results are really good. The status is updated to beta version, Wed Sep 10th.

  • Release Candidate 1 : Mon Sep 15th

  • Release Candidate 4 : Tue Sep 23th

  • Test servers : Razorback (250000 users) & SEDG & OCN & WSL & WWW.NVPHOST.COM & many others

  • WIN32 port (Using native asynchronous winsock calls)

  • 64 bits target machines (IBM AIX V5.1, Tru64 5, HPUX 11, Solaris, linux x86_64, ...)

  • Extension of the UDP protocol to allow clients to ask sources for many MD4 hashes at once, instead of one UDP packet for each MD4. (should reduce UDP traffic by a 2 or 3 factor)

  • Extension of the UDP protocol to send many Search results per packet.

  • zlib support to compress big frames exchanged with clients (share lists, search results)

  • Support of files > 2Go

  • Support of FreeBSD kqueue (FreeBSD equivalent of linux epoll)

  • Support of Solaris/HP /dev/poll

  • Ability to perform a search with MD4Hash (to retrieve associated tags of a file (NAME, SIZE, TYPE, ...)

  • Better server list handling.

  • Tracking of too old emule versions.

  • keepalive pings are most of the time piggybacked into other services messages : lower TCP bandwidth (using less IP packets)

  • ...many more addons...

new New in version .p76 :

2003/07/24 * HOTFIX * A hacker found a way to remotely crash all previous versions of dserver. (SilentBob was attacked today)

2003/07/25 * BUGBUG * The p76 version is buggy, so please wait that I release p77 when all bugs are corrected (this may take 15 days, sorry guys)

new New in version .p75 :

A lot of time since p72, and this web page was not updated. Versions p73/p74 were intermediate test versions installed on some test servers.

  • First : I want to thank the emule dev team for their cooperation to maintain the network in a good health.

  • Successfully validated with linux-2.5.75 and linux-2.6 kernels (epoll, futex, hugetlbpages features) : a bug on futex based mutexes was corrected.

  • One thread is now dedicated to the handling of new connections requests, instead of picking a thread from the common pool.

  • Memory savings, CPU savings, network savings.

  • Corrects a bug that a malicious client could exploit to remotely shutdown the server.

  • Better threads managment. Thread stacks are moved below the text segment, where a 128 Mo hole was unused.

  • Wanted : I *need* an AMD Opteron based machine running linux x86_64 to test the server in a 4 Go virtual space. If someone can give me a ssh access to such a machine... thanks.

  • Blacklisted users are now kicked off the server, instead of their AskSources being just ignored. In previous version, dserver were giving no more sources to blacklisted people but gave their IP:port to other clients. The source exchange done by emule then permit blacklisted client to continue to get new sources without even notice they were blacklisted.

  • Every packet received from a client does cost at least one credit in the slimit module (p58). In previous versions, only the connection request and SourceAsk requests had a cost.

  • When a client is blacklisted, this message is sent before the disconnect :
    "ERROR : your IP is blacklisted because of the behavior of your client."

  • When a blacklisted IP tries to connect to the server, this message is sent :
    "ERROR : Connection refused : your IP is currently blacklisted by the server." Then the connection is closed.

  • When a server is full, a connection attempt results in this message :
    "ERROR : Connection refused : The server is full."
    Note : in this case no credit is taken.

new New in version .p72 :

Several admins asked for this feature. A filter rejects some IP addresses that badly configured servers spread : The list of rejected netblocks comes from RFC 3330 :
0.0.0.0/8
10.0.0.0/8
14.0.0.0/8
39.0.0.0/8
127.0.0.0/8
128.0.0.0/16
169.254.0.0/16
172.16.0.0/12
191.255.0.0/16
192.0.0.0/24
192.0.2.0/24
192.88.99.0/24
192.168.0.0/16
198.18.0.0/15
223.255.255.0/24
224.0.0.0/4
240.0.0.0/4

An old bug is corrected (that bug could lead to cores of dserver triggered by some clients)

The "ExClientDoor Unknown Msg: 162" message is suppressed, since this is a false warning, and emule 0.27 started to actually use the UDP 162 message...

Extended pings now include 2 more integers : The Soft Limit and Hard limit of shared files accepted by the server.

new New in version .p71 : linux 2.5 and epoll ready

  • This version corrects a memory leak introduced in p68. The sort function introduced in p62 was rewritten to be more CPU friendly : On 100k servers, it was taking 25% of CPU.

  • Thread stacks max size lowered from 2MB to 1MB, to free some virtual memory space for future big servers.

  • New thread dispatching. Server is more responsive.

  • A dlaunch program can be used to start the dserver. This launcher tunes the dserver for the target machine :

    • If the target machine has epoll(), dlaunch setup the dserver to use this technology from Davide Libenzi : time for 200k servers is coming.

    • If the target supports futex() syscall (linux 2.5.7 feature), it asks the dserver to use it for mutex operations (thread synchronization). A special rwlock (FIFO readers-writer lock) is used to permit a better CPU utilization (nice on SMP machines)

    • If the target supports HugeTLB pages, it can setup the dserver to use them. (Not the default behavior of dlaunch, since linux 2.5.59 doesnt permit futex() on HugeTLB pages : the kernel just hangs forever, you should use 2.5.60 or Andrew Morton 2.5.59-mm8 to play HugeTLB games)

    • If the target supports SYSENTER (linux 2.5.53 feature), it patches the dserver to use them.

    • If the target machine is an UniProcessor (not a SMP machine), dlaunch patches all "lock" prefixes by "nop".

  • How to use dlaunch :
    Replace : ./dserver
    by : ./dlaunch ./dserver
    in your script.sh or other shell that starts the dserver.

new New in version .p68 : The first to reach 100000 users !

  • Sockets established from the server to the 4662 ports of clients now use IP_TOS LOWDELAY option.
    This helps to get HighID because traffic shaping works even for users that dont use 4662 but other ports.
    Even without traffic shaping, it may help if the routers between clients and server pay attention to QOS bits.

  • The server really enforce the 'Soft limit' : The old servers were limiting the number of shares that were published at once. Some buggy clients were issuing several Publish operation and bypassed the limit.
    The server now sends a message to the user when the soft limit is reached "WARNING : You have too many shares for this server".
    This message can be modified by nbuser : nbuser MSGSOFTLIMIT="Warning, this server enforces a limit of 1000 shares that you seem to excess. Verify your shares or find another server"

  • Better handling of incoming connections when the server is near the maxClient limit. Before p68, the server could accept hundreds of connections (and testing the backdoor ports of clients) even if only one slot was available. Servers should not waste their time/bandwidth in this situation.

  • A new hash table is introduced to reduce the CPU taken by incoming connections : No more whole client list traversal to verify if the IP:port of a requestor is not yet in the list. This was trashing CPU caches horribly.

  • Please consult this document on bigservers if you are the admin of a big edonkey server. It gives some hints that were used on Silent-Bob server to reach 100000 users !

new 2002/12/11 A note about FreeBsd 4.7 :

If you want to run dserver .p67 on FreeBSD 4.7 do the following :
Fetch the cachesize & nbuser programs.
wget http://81.91.66.90/lugdunum/cachesize.gz
wget http://81.91.66.90/lugdunum/nbuser.freebsd.gz
gzip -d cachesize.gz
gzip -d nbuser.freebsd.gz
mv nbuser.freebsd nbuser
chmod 755 cachesize nbuser

Then patch the dserver to tell him not to use inexistant syscalls on FreeBSD :
cachesize -f 1 dserver
Then call brandelf (included in FreeBSD) :
brandelf -t Linux dserver
Your dserver is ready to run. Please use the FreeBSD nbuser version to control your dserver, and make sure that /proc fs is mounted, or nbuser wont work.
Note : with p75, you dont need anymore cachesize program.

new New in version .p67 :

  • Some linux kernels (2.2.19 for example,but other as well) have problem reporting EOF with poll() on sockets sometime. When it happens, the dserver isnt notified that the peer is disconnected.

    Since dserver sends 'pings' to every connected clients every 3 minutes, and relay AskCall for LOWID users, it can queue a lot of messages that are never sent to the frozen connections.

    The used memory just grows and admins may have to restart the whole server when memory hits the limit.

    With p67, the timer now takes care of frozen sockets and disconnects them. The transmit queues are then flushed and memory not leaked.

  • Extended pings. When the dserver receives ed2k pings, with certain challenge values, the answer is augmented with the current 'maxClient' value. Server lists generators (or new clients) could use this information to find servers that have available slots for new clients.

  • Dont blacklist the small connections that got ID=0 (p65 refinement)

  • Disconnects & blacklists clients that publish their files before the server gives them their new ID. (violation of ed2k protocol on old emule versions that costs some bandwidth on servers )

  • nbuser 0.67 changes : If you want to change the slimit cost for short connections (default : 900) introduced in p65, you can use :
    nbuser shortcredit=200 (values between 1 and 1200)
    Please note that ordinary connections already costs 100 credits, so a '1' value means 'I dont care users use old clients and connect on my server (and other servers) just to say "Hello man"
  • If you want to change the duration of blacklist (2 hours per default, 7200 seconds)
    nbuser Blacktime=3600 (duration in seconds)

new 2002/12/04 A linux kernel patch :

It was announced for the .p61 release, it is used on several big edonkey servers with success since two months.

It can save between 20-30% of the TCP Upload of the server.

The rationale of this patch is to delay the pure ACKS so that the server has a chance to NOT send pure ACKS if it can send DATA (with an ACK for free) replies to the client. The default timer in linux is way too low for an edonkey server.

The patch is very conservative and change only the TCP connections to the 4661 port.
It is controlable with 2 sysctls :

net.ipv4.tcp_delack_sport : the port of the server (4661 per default)
net.ipv4.tcp_delack_value : the timer in ms (900 ms per default)

Example of use, for a server using the 4242 port :
echo 4242 >/proc/sys/net/ipv4/tcp_delack_sport

Or you can also add this line in /etc/sysctl.conf
net.ipv4.tcp_delack_sport=4242

The patch is lugdunum_ed2k_delack and can be applied on linux-2.4.19 or linux-2.4.20 like this :
# cd /usr/src/linux-2.4.20
# patch -p1 < lugdunum_ed2k_delack
# make clean dep bzImage ...
and so on you should know how to recompile/install your kernel/modules.
Ask me if you use another kernel. We even patched a linux 2.5.44 with some changes.
Of course, it's better to use a monolithic kernel (no modules) for best performances (modules uses more TLB entries)

new New in version .p66 :

Use a special linux 2.4 setsockopt (TCP_DEFER_ACCEPT) so that the linux kernel gives new connections to dserver only when the client sent some data.

Some cleanups about the login phase of ID=0 clients (ID=0 means that user is a LOWID and the server refuses LOWID)

A timer disconnects ID=0 users that sometime stay connected forever.

New in version .p65 :

Since emule leader (Merkur) said that he wont change its silly behavior (10 simultaneous connections to servers), we decided to make another patch. :(
This version take into account the time of a client connection. If this time is too short, we substract 900 points from the slimit credit of the IP address of the client.
So if emule clients continue to make very short connections to servers, their IP is blacklisted (as were blacklisted BOT and mldonkeyV1 in the past)
Some info on this subject : Why Merkur is plain wrong
Thanks to Tarod for releasing a new Mod (Tarod 9) of emule that corrects this bug.
The next chapter explains the problem we observed on our servers and that Merkur just deny.

2002/11/30 A note about emule :

Emule is now the most used client. This client has some annoying bugs.
The 'Connect' button launch ten connections to ten servers. When a connection succeed, the 9 others are closed. The effect of this tactic is that servers are now flooded by incoming connections that last few seconds (between 3 and 10 seconds).
This tactic is not very good, because chances are great that servers test the 4662 port of the client at the same time, and give LOWID to client because of timeouts or port unreachable. We asked Merkur to change this silly behavior but got no answer for the moment.
In the mean time, servers admins have to increase the number of threads.
Example : Silent-Bob uses 200 threads for 50000 clients.

2002/11/15 A note about emule :

Old versions of emule have a lot of bugs. Some users continue to use them and harm the whole network.
For example, servers do receive a lot of UDP traffic to wrong ports.
Very old versions of emule use UDP 4665, some other use UDP 4 port, instead of the (TCP+4) port.
You can blacklist those clients using the 'recent' netfilter module :
DROPing the frames is better, because your machine wont send ICMP messages saying that UDP ports are not reachable, since emule dont care at all about ICMP errors. Better to save your bandwidth.

iptables -A INPUT -p udp --dport 4665 -m recent --set --name badguy -j DROP

iptables -A INPUT -p udp --dport 4 -m recent --set --name badguy -j DROP

iptables -A INPUT -m recent --rcheck --second 7200 --name badguy -j DROP

iptables -A OUTPUT -p icmp --icmp-type port-unreachable -j DROP

Of course if your server use the standard 4661 port for TCP, the UDP is 4665, so dont use the first rule in this case !

New in version .p64 :

  • A bug on Searchs with types 'Video', or 'Audio', or 'Images' was introduced in p63, and corrected in p64
  • Some numbers :
    a p63/p64 dserver with 5700 users 265888 files uses 87 Mo of RAM
    a p63/p64 dserver with 16626 users 803792 files uses 246 Mo of RAM.
    a p63/p64 dserver with 28376 users 1480244 files uses 382 Mo of RAM.
  • Clarification : A Pentium+ CPU is a CPU compatible with Intel© Pentium CPUS. So an AMD K5 / K6 / Duron / Athlon are OK ! In fact nearly everything shiped in the last 5 years is OK. Unless you bought a Sinclair ZX80 :)

New in version .p63 :

  • ** Warning ** : This version requires a Pentium+ CPU to run, because we use cmpxchg8b instructions, that let us use special lock less algorithms.
  • When allocating memory, we try to allocate hugetlb pages (4Mo pages) that linux 2.5 provides. The syscall used is sys_alloc_hugepages(), and has no effect on old kernels. GRUK server uses linux 2.5.42 without problem. You dont need linux 2.5 to run, you can stay with linux 2.4 of course.
  • A thread is dedicated to the handling of UDP messages, instead of the pool of threads. This result in an ultra-fast handling of UDP messages : pings delays are very small.
  • The server can use a community string : Users are allowed to log only if their pseudo contains the comunity string.
    This facility will help private servers to stay private, even if some edonkey clients exchange their servers lists.
    Example :
    We want to allow logins of clients having this string in their pseudo : [afpteam]
    nbuser -c "[afpteam]"
    To unset the community string, use : nbuser -c ""
    To conserve their limited bandwidth, admins of private servers should consider blocking UDP traffic using the firewall :
    iptables -I INPUT -p udp --dport 4665 -j DROP
  • The server does a more complete test on the backdoor of the client that wants to connect :
    We verify that the port 4662 (or the port specified by the client) is indeed mapped on its edonkey client and not a fake application. If it is the case, a LOWID is given.
    Emule clients may have some problems about this detection, Merkur had been informed and the bug should be corrected in the next Emule version.
    A workaround for this problem is included (small delay between the connect and the sending of the data on the socket)
  • Please consider to use a higher value for 'threads' parameter, or your server wont be able to accept a lot of new connections (bursts of incoming clients).
    This is advised because of the longer time the login process takes due to extended port 4662 probe.
    Examples :
    • lugdunum, 5700 users, threads=50
    • gruk, 30000 - 50000 users, threads=100
  • A memory leak had been corrected.
  • Smart Sources parameters updated : 200 sources max in an answer to a TCP request, 50 in UDP. Due to limit per source (per unit of time), it concerns few requests.
  • We are planning to use /dev/epoll linux facility from Davide Libenzi to be able to handle 100000+ clients. So, if you want to anticipate our next release, you can already try to install the package suited to your kernel.
  • Jari, an edonkey server admin had problems about iptables connection tracking overflows on his linux machine (also used as a NAT router for his internal network). We suggested him to use 'iptables -A INPUT -t mangle -p tcp --dport 4661 -j ACCEPT' so that edonkey connections are not going through connection tracking module, wasting a lot of memory. Jari reported that this suggestion works very well. It might help other admins to try this.

New in version .p62 :

  • New hash table for clients lookups. Servers accepting LOWID users should gain a lot of CPU.
  • Highly optimised substring searches, to speedup Searches, especially on large servers.
  • We sort the lists associated to keywords, so that 'Search Requests' delivers files having the highest availability. Users should be happy !
  • High end servers should handle 60000 clients with this version. GRUK can not because of it's too small Internet connectivity. Anyone wants to test ?

New in version .p61 :

  • Optimisations... Again... Huge CPU savings, and RAM savings.
    GRUK server handles 40000 users with this version of dserver. (well... 39950 because we forgot to change the ulimit -n 40000 in the script)
    GRUK handles 50000 users with a linux kernel patch (on top of linux-2.4.19) that will be released in a couple of days. This patch tries to delay as much as possible TCP ACKS sent by the server : They can be canceled if the server has some data to send.

New in version .p60 :

  • Oops... a memory leak was introduced in p59. This should fix it.
  • A correction has been done to enable dserver running on gentoo linux and kernels that use a Kernel/User split of 3G/1G instead of standard 1G/3G.

New in version .p59 :

  • Huge RAM savings... Again...
  • Use of the setsockopt(TCP_QUICKACK disable) to try to delay as much as possible TCP ACKS, in order to save Upload on our servers.
  • We asked Merkur of Emule project to help us in saving Upload : When a client sends a frame to a server, it is currently using two IP frames. It is easy to coalesce the two frames onto one frame only.
    This kind of client optimization could let the server save 15% on Upload.
    We are working to coalesce UDP requests too. This would save much more bandwidth, but it's much more difficult to do.
  • small bug correction : dserver was core-dumping when the admin wanted to quit the programm.

New in version .p58 :

  • Better handling of incoming TCP connections.
  • Slimit module integrated in dserver :
    • This module keeps track of the requests coming from each IP address.
    • UDP messages, TCP FileIdSearchs, TCP connections use slimit module to detect BOT and abusers.
    • When an IP triggers the slimit module, it is blacklisted for one hour.
    • Users doing connections that last small time can trigger the slimit module.
    • Users trying to download more than 60 files in // can trigger the slimit module.
    • Protection of dserver against Denial Of Services attacks (DOS)
  • Better handling of idle threads : They sleep for longer time, reducing the load of the machine.
  • Gruk server can handle 30000 users with this version of dserver.
  • Well configured machines (SMP, 1Go RAM), with broadband internet access can probably reach 40000 users with this version. (Try threads=50 in donkey.ini for high end servers)

New in version .p57 :

  • Smart Sources V3 : Detection of BOT and anti social modified edonkey clients. You are going to love it.
    Our analysis show that on our test servers (lugdunum, gruk, ed2k.ch, link92 and fugitif), 1% of the connected clients use the BOT or mldonkey. They were 'stealing' between 30% and 50% of all sources delivered by the servers !
    With Smart Sources V3, they are detected and the server send to them only 1% of the sources.
  • As the logfile of the server is updated, we may develop a daemon which could dynamically add firewall rules in order to blacklist offensive IP addresses for 24 h and share the information with other dservers.
  • Please note that we work with mldonkey author to make his programm much more friendly with servers.
  • Thanks very much to the sysadmins of ed2k.ch (bile666), Gruk (Corpo), Link92, Fugitif67 for the help they provided to track BOT clients behavior.
  • The Smart Source parameter (TCPmax) is changed from 20 to 40 sources max included in an answer to FileIDSearch.
  • Some bugs corrected - We urge you to update your old versions (< p56)

New in version .p56 :

  • Special provision to allocate structures with an alignment multiple of L1_CACHE_SIZE (The size of a cache line in the processor cache)
    We use a special memory allocator in order to make sure only one cache line is referenced when accessing hot spots.
    The standard libc malloc() enforce an alignment of 8 bytes, which is not sufficient at ALL.
    L1_CACHE_SIZE is :
    • 16 bytes for i386 and i486
    • 32 bytes for Intel PII and PIII (default shipped in dserver-16.38.p56)
    • 64 bytes for AMD Athlon
    • 128 bytes for Intel PIV
    If you want to adjust the value (default 32), please use "nbuser -C newvalue"

  • If two or more edonkey clients share the same IP address, but use differents Door ports, they can now get HighID.

  • A message is sent when a LOWID tries a connection to the server. It can be :
    • WARNING: your IP X.Y.Z.T, port 4662 is already connected to this server.
      This message is sent when the server detects a second connection with same (IP,port). It can happens if two edonkey clients share the same IP and they both use the standard Door Port value (4662) It may happens if a network disconnection has not yet been propagated and acknowledged.
    • ERROR: Your port 4662 is not reachable. You have a LOWID http://www.edonkey2000.com/documentation/lowid.html ! Please correct your network config !
      This message is sent if the Client's Door port (default 4662) is not reachable.
      We hope the message will be read by newbies.

  • LOWID are BACK ! We added a limit based on the percentage of LOWID users. The default value is 20%. Change with nbuser -R XX. The success of Smart Sources V2 makes LOWID acceptable on servers because they generate much less traffic than before.

  • If a client has in its pseudo the string "nolistsrvs" (case not significant, it can be "NoListSrvs" ...), the server doesnt send its servers list and its description when the edonkey client ask it. This lets the clients decide themselves of the content of their server list.

  • More sensible vc , kill, ip commands. vc command displays for each client an unique number, guaranted not to change. ip and kill command now take this number as a parameter.
    vc command displays the ID and IP:port of each client.

New in version .p55 :

  • Huge speedups. Gruk French edonkey server handles 20000 users instead of 12000. This version is a must for big servers if they were CPU bounded.

New in version .p53 :

  • Smart Sources V2 : Refinement of Smart Sources, based on feedbacks we got from users.

New in version .p52 :

  • Original server uses only the 10 first words in a file name to index it. If a file was named "a big tree in a little forest, ripped by the avenger.avi", a search on "avenger" could not find it. This version put in the indexes all the words of the filename.
  • Dont index one letter words : it's just useless.

New in version .p51 :

  • A bug about nbfile is corrected. Please dont use .p49 or .p50.
  • Dont send LOWID sources to other LOWID users
  • vc command shows in S: column the number of time the client was given (as a Source) to a FileIDSearch requestor

New in version .p50 :

  • A bug introduced in .p49 is corrected. We didnt answer very well to 'File Searches' UDP.

New in version .p49 :

  • You wont believe it. I mean it.

  • Smart Sources enabled : This an exclusive lugdunum feature.

    'Smart Sources' is a special handling of 'FILEID search' requests.

    • Avoids sending the same sources (ID) over and over :
      Implements a rate limit in order to avoid sending IDs that are probably already too busy with uploads.
    • With this smart evolution, LOWID could be accepted again on our server because they probably would generate less traffic.
    • Reduce the inter-client traffic : When a client received 10's incoming connections per second, 99% of them were thrown away.
      Since we reduce the overhead of the edonkey network, the bandwidth of transferts will increase.
      More information about 'Smart Sources'

  • CPU needed divided by three, thanks to having a list of shares per client.
    The old versions needed to scan all the Files of the server to clean the files shared by one client. This had huge impact on performance on big servers.
    You are now able to choose higher values for the donkey.ini tableSize parameter, without performance drop ! We recommend tableSize=65029 (or bigger) for 5000 users servers.

  • "vc" commands shows p:number of shares for each client

New in version .p45 :

  • The nbuser command displays UDP bytes counters

New in version .p44 :

  • The server disallows mldonkey (MLD) users login by default.
    To allow them, please use "nbuser -Z 1"
    nbuser prints information about MLD :
    ...
    Anti MLD filter enabled. 88 MLD connections rejected
    ...

New in version .p43 :

  • Version .p43 is compatible with the 'linux-mpss' patches that allow us to use hardware pages of 4Mo instead of 4Ko.
    This mpss patch applies on Pentium+ CPUS, capable of PSE. (PSE : Page Size Extensions)
    It reduces TLB pressure by a 1024 factor ! It really helps programs that use more than 16Mo of memory, especially if memory is fragmented.
    linux dservers will be VERY happy to use mpss (Current mpss patch available on top of linux-2.4.18).
    dserver .p43 allocates memory using mmap() calls with blocs of 4Mo, followed by a call to madvise() as documented in the mpss docs.
    Of course .p43 and up versions are able to run on a standard linux machine, not mss patched !

    Since August 14th, lugdunum ed2k server uses linux 2.4.19 + Super Page patch from Naohiko Shimizu.
    Super Page patch is an alternative to mpss and seems quite robust. The fact that it is available for linux 2.4.19 makes it a better candidate than mpss for the moment.

New in version .p41 :

  • nbuser -F display counters of matches of filters[] that JED documented here.
    I suggest to use :
    filter[0]=(#FILENAME .part)
    filter[1]=(#FILENAME .met)!(#FILENAME maurice)

New in version .p40 :

  • The welcome[] frames are sent with one TCP message, instead of one message per line. It saves some bandwidth on the server.

New in version .p38 :

  • More optimizations and cleanups before the big changes (Smart Sources), servers should benefit a 5-10% gain of CPU against .p37
  • The vc command (View Clients) has been augmented to print the counters of xmited and received bytes for each client/user.

New in version .p37 :

  • More optimizations and cleanups, servers should benefit a 5-10% gain of CPU against .p36

New in version .p36 :

  • An incredible bug in the original dserver has been corrected : It didn't handle correctly EOF on sockets : When a client disconnects, the call to recv() returns 0 (EOF) and the dserver didnt notice about that !
    Correcting this bug will lead to less LOWID connections, because when a user disconnects and re-connects shortly, the old server was giving a LOWID !
  • More optimizations and cleanups, big servers should benefit a 5-10% gain of CPU against .p35

New in version .p35 :

  • Only one send() syscall instead of two : better compatibility with old linuxes and OpenUnix8 (no more use of MSG_MORE flags requiring linux 2.4.4 & up)

New in version .p31 :

  • Ability to allow LOWID connection based on the edonkey pseudo. Put in a file "edonkey.access" some lines with "pseudo:1", and use the command "nbuser -A edonkey.access" to make a hash table and load it into the running dserver. Now you can allow LOWID for guys who contact you and explain why they couldn't get HighID.
  • Ability to control the public flag with nbuser command.
  • Ability to change the number of sources included in a reply to "FileID searchs" UDP & TCP requests. (default 230)

Work in progress

  • A windows version of the patches. With only highly demanded features, not heavy optimizations. The first version is available here and it forbids LOWID connections. It doesn't make an exception if your IE has a HTTP proxy configured.
  • We study the possibility to have a black list of file ID (hashes) you dont want your users share.
  • (canceled)A future version will be able to filter *.met and *.part.met that lazy users shares (because they share the edonkey temp directory).
    It is not needed anymore since JED documented the filters.
    Just add this line in your donkey.ini:
    filter[0]=(#FILENAME .part)

We suggest the nbuser tool to sample various statistics about the running server, and also to change dynamically some parameters.
FreeBSD users should take this version : nbuser.freebsd.gz
$ gzip -d nbuser.gz
$ chmod 755 nbuser
$ cp nbuser /usr/bin # (only if you are root)
$ nbuser
Patch version p58
5536 users 245798 files (146128) avg=44.3999
Start Time : Fri Sep 20 10:10:27 2002
MaxClients=5700
33452 refused connections because of MaxClients
minVersion: 57 maxVersion: 2000
 62 rejected connections because of minVersion
tableSize=65029 public=1 threads=40 (40 sleeping) console=true verbose=true thisIP=62.161.204.20:4661
  maxSearchCount=200 maxUDPSearchCount=20
Every 2*10 seconds, timer does a ping to 1/10 of clients (idx 7)
delay between 'Finding other servers' : 3600 seconds
malloc8() stats : 28 4Mo blocs allocated, 157120 bytes remain in heap. L1_CACHE_SIZE=64
Share limit : 1000/client. Hard limit : 4000/client
197 clients had more shares:
 total shares = 490913, average = 2491.94 Max number : 14846
20 clients kicked off because of hard limit
18 disconnections because of a too long frame
Anti LowID filter disabled (1).
 1103 LOWID. Max percentage of LOWIDS 20 %. 7731 LOWIDS rejected because of max percentage.
 2414+0 LowID connections rejected
Anti BOT filter disabled
Anti MLD filter disabled.
TCP stats :
  44693 connects, 29810 ServerList, 122039 Publish, 27149 Search,
  2357593 SourceSearch, 104 UserSearch, 3611188 AskCall, 284 Unk9,
  660 cursorsearch,
  126649 (8.30966 %) refused files (because of donkey.ini filter[])
  6781156 TCP messages sent
UDP stats :
  717407 Ping req 47297 Search req 9040908 FileID search 331247 Ask LowID
  129 Search User 2993 ListServ req 448 ListServ reply 3877 ServDesc req
  175 ListServ ask
  717407:10043698 PingReply 384026:51564836 SearchResult 3033651:96312369 FileIDResult
  32547:195282 UnknwnID 175:63573 ListServReply 3877:209358 ServDescReply

4171683 UDP messages sent. 158389116 bytes (not including UDP headers)
275196240 bytes (including headers) rate : 4898.91 bytes/s

SmartSources: TCP : max=40 nb=6S/60sec 13627590 hits 11896109 misses 1648111 scan 90773 reorg
SmartSources: UDP : max=8 nb=4S/90sec 6445500 hits 104189345 misses 6233054 scan 71916 reorg
slimit: 4829/131072 slots, 12172533/13628869 frames (10.6857% rejected) 1797 accept() refused

To modify the maxClient parameter to the new value 1200 :
$ nbuser -m 1200
To enable connections of LowID clients :
$ nbuser -l 1
To disable connections of LowID clients :
$ nbuser -l 0
To enable connections of BOT-lite clients :
$ nbuser -B 1
To disable connections of BOT-lite clients :
$ nbuser -B 0
To enable connections of mldonkey clients :
$ nbuser -Z 1
To modify the soft limit (max number of shares) to 90 (default : 500) :
$ nbuser -s 90
To modify the hard limit (to kick off lazy edonkey users) to 2000 (default : 4000) :
$ nbuser -D 2000

lugdunum uses the superlimit (from Joakim Axelsson) netfilter patch, which is not yet in the standard netfilter package. This filter ignores the frames coming too rapidly from one IP address. This is caused by users downloading too many files, and generating too many 'File ID Search' to the edonkey servers.

We strongly suggest to use 'traffic shaping',
to control your bandwidth, install shaped in /etc/rc.d and add this line at the end of /etc/rc.d/rc.local
/etc/rc.d/shaped
Before using it, you will probably have to change the variables defined at the begining of the file : UPLINK, LOWUP, DEV and LOCALNET
If you have a local network and have configured your linux machine as a Masquerading gateway, you can ask that traffic coming from local network be in High priority class. Even if your edonkey server is running, you will be able to surf the Internet from a PC on your local network :
If for example you have an internal network on the eth1 interface, you mark the paquets incoming to that interface with this command :
iptables -A PREROUTING -i eth1 -t mangle -j MARK --set-mark 2
If you use a PPPOE line with a MTU of 1492, we suggest also :
iptables -I FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu

We suggest to install /etc/rc.d/adjust an add this line at the end of /etc/rc.d/rc.local :
/etc/rc.d/adjust
This script changes some system parameters in order to have an optimal edonkey server.

For big servers, dont forget to increase /proc/sys/fs/file-max. (default value 8192).
You can add (or modify if already present) this line to /etc/sysctl.conf :
fs.file-max=16384
You can also do : echo 16384 > /proc/sys/fs/file-max

On linux 2.2 machines, dont forget to change /proc/sys/net/ipv4/route/max_size, because a too low value can explain some problems.
echo 32768 > /proc/sys/net/ipv4/route/max_size
You may also change the linux 2.2 VM settings :
echo "1024 2048 4096" > /proc/sys/vm/freepages
Linux 2.2 has a limit on the poll() system call : Because it uses the kmalloc(nbhandles*8) kernel service, it has a limit of 16384 file handles. So a dserver running on a linux 2.2 machine can not have more then 16300 users.

Last advice : we suggest to launch the program with a script script.sh : It uses ulimit -n 8192 to change the maximum number of files a process can open. If you plan a 10000 users server, change this to 16384 !

For users wanting to run dserver as an ordinary user (not root), you may change /etc/security/limits.conf and add this lines :
* soft nofile 32768
* hard nofile 32768

We provide a modified version of tcpdump (network sniffer) to decode edonkey protocol UDP 4665.
Usage :
# tcpdump -p -n -s 1600 port 4665
17:48:53.199825 80.15.37.251.1095 > 62.161.204.20.4665: ed2k FileID search hash=4938BC08FC797F0ED9007AD3286E3998
17:48:53.217966 62.212.110.223.2297 > 62.161.204.20.4665: ed2k FileID search hash=8C6E353331519603C2217A6C9070B98B
17:48:53.218973 24.202.51.101.1041 > 62.161.204.20.4665: ed2k FileID search hash=43D12BBDE83FC8BDF112330F463931E9
17:48:53.220204 62.161.204.20.4665 > 80.15.94.151.2390: ed2k Ping reply challenge 119422780 nbuser=4492 nbfile=440958 (DF)
17:48:53.223180 62.161.204.20.4665 > 81.65.55.10.3037: ed2k FileID result hash=1F9E0F75A78EB744BEFC3A71B204493C 108 sources (DF)
17:48:53.225722 80.14.220.19.4628 > 62.161.204.20.4665: ed2k Ping req challenge 3469
17:48:53.225935 202.175.91.40.1025 > 62.161.204.20.4665: ed2k Ping req challenge 5400597
17:48:53.226148 80.11.127.71.3314 > 62.161.204.20.4665: ed2k FileID search hash=FB997EF2ECE60053C665816D245DBB73
17:48:53.247157 62.161.204.20.4665 > 213.44.161.69.1085: ed2k FileID result hash=B99128E049F681E607265BD078EB1E32 48 sources (DF)


EMail