NAT Traversal Constraints

Whilst NAT traversal as implemented by DC++ has proven capable of rendering many formerly mutually inaccessible pairs of passive users capable of directly connecting to each other, it has limitations arising from the TCP gimmickry used to implement it. This post explicates why certain people will likely see failures if they test their NAT traversal compatibility.

Many TCP NAT traversal methods exist, but DC++ uses P2PNAT. The limitations and specific methods discussed here have alternatives, but largely less convenient alternatives. According to the 2005 paper Characterization and Measurement of TCP Traversal through NATs and Firewalls, P2PNAT is one of the only options not requiring superuser privileges, does not require raw sockets, and does not risk timing races; as a result, the its faults and limitations herein characterized still leave it as the preferred algorithm:

The most obvious such P2PNAT limitation, and one relatively easily ameliorable, involves NAT traversal users attempting to download from each other. As presently implemented, those antiparallel connections cannot be formed due to sharing the same 4-tuple of (source IP, source port, destination IP, destination port) and as such the operating system TCP/IP implementation cannot distinguish between them. Ordinarily this is not a problem because when establishing a connection to a remote machine, the initiator binds to any local, ephemeral port that might be available; thus, even if the destination IP and destination port are shared between connections or the source IP and source port are shared between connections, other members of the tuple provide the necessary uniqueness. However, the NAT traversal method DC++ implements depends on controlling this all four members of this tuple – the source and destination IPs and ports – on both sides of the connection. Therefore, it deterministically attempts to set up the same four-tuple for each user twice when two users simultaneously attempt to connect to each other. Because TCP/IP implementations demand that this tuple be unique, they reject the second connection attempt, so only one DC user in a NAT-T’d pair can transfer from or to the other at any given time.

This effect lingers, interfering with even non-simultaneous connections due to a byproduct of the TCP state machine. Whilst the C-C connection exists, both ends are in the green “ESTABLISHED” state. However, one side then actively closes the connection, at which point it goes into the FIN_WAIT_1/FIN_WAIT_2/CLOSING/TIME_WAIT state region, whereas the other client sees its connection get passively closed and enters the CLOSE_WAIT/LAST_ACK state region. The latter has no problem: the connection is gone and such tools as netstat will cease listing it. However, the former, actively closing machine gets stuck in TIME_WAIT for twice what’s called the maximum segment lifetime, which can come to as much as 4 minutes. One can reduce the TIME_WAIT interval both in Windows and Linux, but it protects against a stray, resent packet from a previous connection from breaking established TCP connections, so it’s unwise to reduce it exessively. Further, it’s a system-wide setting in both instances, so DC++ should not change it unilaterally.

During these long TIME_WAIT-delayed minutes, any new attempt to re-establish a connection between two NAT-T DC clients which had just closed a connection will fail, because one of them will associate that unique (source IP, source port, destination IP, destination port) tuple with TIME_WAIT and thus prevent the new connection from forming. Fortunately, the same ameliorative steps which reduce the incidence of this issue for simultaneous connections render this unavoidable TIME_WAIT less problematic.TCP state diagram

This simultaneous connection and near-successive connection limitation is fixable by creating more local ports to which the NAT traversal code can bind, so limiting as it may presently be, it’s not an intrinsic problem with NAT traversal. Other, more inherent problems do exist and can be caused by operating systems and routers. The NAT traversal compatibility survey aims to discovering the prevalence of such problems.

Windows 2000, XP SP2 losing MS and DC++ support

Win2k leaves its extended support period on July 13 this year, the same date that WinXP SP2 is retired. To receive further official security updates, Windows 2000 users must update to a newer system and WinXP SP2 users must update to SP3.

When even Microsoft stops supporting one of its systems, official DC++ support ceases as well. Both Win2k and WinXP SP2 remain similar enough to the still-supported WinXP SP3 to likely remain compatible with DC++, but bug reports filed from systems running operating systems older than WinXP SP3 may now be ignored; and future changes which undermine support on those systems may be implemented regardless.

NAT Traversal Compatibility Survey

Search support for NAT traversal has landed in the DC++ trunk. It’s not yet in any unstables builds, but it should show up in Toast’s ADCPortal build archive shortly as DC++ revision 2160 or later. Passive mode automatically enables NAT traversal; there’s no explicit setting for it. If it works, two passive users are able to communicate directly with each other via a client-client TCP connection and search and transfer files between each other.

The DC++ developers are interested in the success or failure of NAT traversal depending on individual conditions. In particular, router model and operating system have a large impact. Thus, anyone trying this feature (it requires that both involved clients support NAT traversal) is encouraged to describe their results in the DCDev Public hub at adcs://devpublic.adcportal.com:16591 or the ADC Portal forums with their operating system, router setup, and degree of success.

Consolidating Mods (or, a Plugin Interface)

At least 8 current maintained DC++ mods (BCDC++, StrongDC++, ApexDC++, and all of StrongDC++’s other immediate derivatives) exist which largely modify some cosmetic aspect of DC++ whilst keeping the core unchanged.

Some, but not all, of their functionality could be implemented via a plugin interface which would allow many users of those mods to instead use the official DC++ client. This has advantages in terms of:

  • bug reporting (rather than users in each client having to independently discover bugs, bug reporting could be more centralized)
  • bug fixing (rather than a choice of either each mod author separately patching his client or syncing with DC++ when it might not be appropriate)
  • security responsiveness (an important special case of bug fixing; especially a problem for second or third-tier derivatives which need to either wait for fixes to propagate down the mod dependency graph, remain vulnerable, or specifically re-implement some fix that will automatically appear for them in DC++ regardless)
  • security improvement, if features already extant in DC++ can be reimplemented in a safer scripting language as opposed to C++ (Python/Perl/Lua/etc all avoid entire classes of bugs by default which C++ code can be vulnerable to)
  • ease maintenance of somewhat fringe/marginal features which some people enjoy (for example, now-playing song/music spamming) but which aren’t necessarily appropriate to maintain in the core DC++ application
  • maintaining a DC++ brand image (every mod doesn’t have to be identical, but because each mod can lag, it can force users into unpleasant choices, which can hurt the DC++ brand)
  • user control of clients (some changes can be easy for technically sophisticated end-users to create and distribute, but again are not always appropriate for all users)

One such proposal, which I essentially support, has been implemented by Crise and proposed by him and Toast.

File corruption vulnerabilities in DC++ and based clients

The latest stable release of DC++ fixes a long standing problem in error handling during downloads. This bug can cause file corruption, crashes and even a moderately critial security vulnerability. Most probably the problem exists in all DC++ versions released since Tiger based integrity check and file identification has been introduced. This also means that the bug should affect all clients based on DC++ libraries older than version 0.762.

In short, the problem arises when any kind of errors happen during downloading of the last chunk of data of a file (the data corresponding to the last TTH leaf  of the Tiger tree). The error handling is incorrect in this case and despite the actual error the download will be finished without any warning. Everything happens like in case of any successful download: the corresponding queue item will be removed from (or depending on the setting marked as finished in) the download queue as well as the (likely corrupted) unfinished file will be moved from the unifinished downloads to the finished downloads folder.

However, the queue item removal is not done properly in this particular case; this can lead to (not necessarily immediate) crashes or freezes, especially in the newer DC++ versions. So above the possibility of getting corrupted files marked as successful downloads, if you download from a malicious client that produces specially crafted data flow it can cause further instability or even DOS.

Additionally, this problem can be used to spread fake files over the DC network since when none of the download sources are able to produce a full tiger tree for the downloaded file a special spare integrity check is done after the download finished. Without having the full Tiger tree, integrity check is done by TTH only after the file has been fully downloaded. Because of the same bug described above, even if  this TTH check fails, the downloaded file will be saved to its target location without any indication of errors.

This means that searching for a known trusted file by TTH is not safe in DC as long as you don’t use the latest client versions. Rogue clients can easily create fake file lists or respond so they share the file you searching for (and known harmless by TTH). If these malicious sources refuse the query for the full Tiger tree then any file of the given size but arbitrary content will be accepted from them without showing any error message. Successful exploitation needs that no other sources provide the correct full tree before the download starts.

This problem exists in almost all clients based on TTH capable versions of  DC++. The oldest tested and vulnerable version is DC++ 0.4032. The bug can be reproduced with various clients using current and former download manager implementations (the original DC++ single source, RevConnect based multi-source and current DC++ multi-source are among them). Some client versions like  StrongDC++ 2.21 and above (and its derivatives) are the only exceptions; these clients do not accept files with wrong TTHs due to tentative fixes made by Big Muscle to avoid this problem (but they still can crash the way written above).

Client versions contain fix for all the three problems are DC++ 0.762, StrongDC++ 2.41 and any client based on these. As always, upgrade is recommended as soon as possible.

DC++ 0.762

DC++ version 0.762 has just out. This release mainly focuses on security and stability updates, fixes a few annoying bugs that came with 0.76x series and brings you a few smaller improvements as well.

Fixed bugs :

  • Fixed random problems with hidden files, folders or complete drives (root folder) cannot be added to share or they don’t appear in the file list even after hashing was finished.
  • Fixed a problem with downloaded filelists : they weren’t correctly positioned when opened and they could get stuck in the download queue in certain cases.

Improvements :

This release also fixes a security hole found in OpenSSL, allowing malicious users to remotely crash DC++ so immediate upgrade is recommended for everyone.

As usual you can read the changelog for complete list of changes.

DC++ 0.76x improvements in detail – Part Three: Behind the Scenes

As you may know, the structure of the source code of DC++ is divided into many parts. Basically it can be grouped into three major parts, as the GUI, the DCPP library and additional open source libraries. We covered the most important changes in the GUI part in the first and second post of this series. We won’t go into details about what’s new with the additional libraries since they are (mostly) developed by others.

So let’s talk about what’s happened around the DCPP library (or DClib) recently. DClib is the heart of  all  DC clients which inherit code from DC++. This means that the changes we discuss in this post are not unique to DC++: all the DC clients shown in the link above will incorporate most of the improvements as well as the compatiblility changes sooner or later, as they release their new versions.

The new version fixes two long standing and annoying problems with hashing files. As we know, if you add files to an already shared folder and want to hash them immediately you select Refresh filelist. However, in most cases after a simple refresh command the new files wasn’t added to your own filelist so you needed refresh twice. This problem is hopefully gone now, along with another annoyance when you wanted to restart DC++ while a large file has been hashed. In the latter case the the thread busy with hashing was not closed immediately with the rest of the application, keeping the user from restarting DC++ sometimes even for several minutes.

A memory leak happened in (automatic and manual) refreshing of the filelist is also fixed. It is an important fix to those who are with large shares and running the client all day. These users often experienced that after a few days of uptime all free memory was eaten out by DC++.

From now you must experience much less UAC warnings under Vista or Windows 7, since all keys are written to per user sections of the Windows Registry.

An identifying problem of users in ADC hubs has been solved which could happen when two users both were logged on to (two or more) same hubs. Connection requests, searches and private messages could went through other hub than selected/used which could cause privacy problems.

User commands support for ADC is greatly improved and modified according the current protocol standards.

The code for parsing XML files was completly rewritten to meet the current XML standards and to avoid various attacks trying to distract the normal operation of the program by providing malicious input files. As an effect of this change, all not fully XML compatible hublists will be rejected by the new DC clients. Fresh hublists provided by major hublist servers do not have this problem but some small static hublist files or old / cached hublists may fail to open from now. This applies to all old non-XML format hublists (their name usually contain .config) as well, because upon this change, support of them is also removed.

As you already informed before this version of the lib also drops compatibility with clients using ADC version 0.10 and earlier.

The list of changes is now over, but there’s more: a lot of small things you can be informed from the changelog. We hope you’ll like all the advantages of this year old development… and beware of another milestone improvement coming soon :)

DC++ 0.76x improvements in detail – Part Two: Here and there

Above the major GUI changes mentioned in the first part of my series of posts, there are many more things added to improve the user experience in DC++. We will run through most of the useful new features, a few of them need only a short mention while some others worth a brief explanation.

Let’s start with the toolbar which is, above the visual changes, also more configurable from now. In other words so you can set what icons would you like to see there and in what order. To open the Toolbar Customization dialog you can double click to the empty area of the toolbar or right click and select Customize from the context menu. The usage of the dialog is pretty straightforward and has its own help page in case of any questions. In the context menu you can also set the size of the toolbar icons for your best needs.

Its possible again to hide the toolbar, the transfer window and the main window’s status bar as in the older versions with the same menu items (in the Window menu) and hotkeys just like before.

File hashing can be paused at last, just like in other DC clients. Its a pretty useful feature when you want to stop hashing for a while to temporarily free up all resources for another task. You can pause and resume hashing by a new button in the Indexing progress dialog.

A new possibility added to keep already downloaded files in the queue for informational purposes. The function is not complete yet and  its meant to be an alternative to the Finished downloads window later. You can try it if you enable it in Settings/Queue/Keep finished items in the queue. You can read more of its current behaviour in the context sensitive help (as usual).

While we are at the context sensitive help its worth to mention that the Settings window introduces a new innovative solution as the help text for the currently selected control is always shown in the bottom of the window giving an always visible hint for the uncertain user. There are other news about the help, it has been improved a lot again and has been almost fully translated to 3 languages (French, Hungarian and Polish) and its in partially translated state for many other languages as well.  We introduced the web version of the DC++ help some time before and with the release of 0.760 the webhelp contents were also updated. Its available for anyone at http://dcplusplus.sourceforge.net/webhelp and can be a good info base for any DC network members, not just for users of  DC++.

We already covered the new bandwith limiting function, another visual addition for the limiter is that the current up and download limiting value is always shown in the rightmost part of the statusbar.

If  you check the Connection settings panel, you’ll find that the connection options are completly rephrased. We hope that they will give a better clue to the users about the actual meaning of the options. This is important for hub operators or anybody who gives support for DC++ users (if you do not run DC++ you can check the webhelp link above to see what’s changed). The context help for connection options was also rewitten to be more understandable.

Many new hotkeys added to easily navigate between windows and reach functions quickly. This is really good news for visually impaired users as from now their experience should be much better using DC++. You can check the Help file for the list of all available shortcuts.

You’ll see a very unique new behaviour if you type a pm or mainchat message longer than the available space in the input box. Check it out you’ll like it :) You can configure it in the Settings/Appearence panel.

There are improvements in the public hublist window as well. From now you will get better messages about how fresh is the shown hublist, was it a direct download or is it an older cached version. Also the usage of the hublist Filter box made better introducing filter-as-type behaviour (this also applies to the filter below the userlist in the hub window).

And at last some good news for those who usually run DC++ minimized to tray but don’t like sound alerts:  the DC++ tray icon will chang to letter shape when you get a PM…

That’s it for today, we hope you’ll find the new DC++ GUI a lot better, enjoyable and usable. Soon, in the next and concluding part of this series of posts I’ll write about the changes in the DC library.

DC++ 0.76x improvements in detail – Part One: GUI reloaded

When you upgrade to the 0.76 series of DC++ you’ll immediately see the new visual improvements: nice, modern looking  icons in the toolbar, the clearly better way of how tabs, menus and other GUI parts obey to the used Windows themes. We already mentioned these design improvements in a previous quick run-through post of new features.

However, above the fresh new look, there are equally exciting functional changes in the GUI that I will discuss in this post. The heart of the biggest GUI functionality changes is called WindowManager and basically its made to keep track all the opened windows in DC++. Using the window manager, the list of the opened windows is kept even across sessions and it made possible to implement a few really useful functions in DC++.

If you start DC++ the first time after upgrading to 0.76x you may won’t have some of your usual windows (like download queue or favorites) auto-opened. If you check the Appearence/Windows settings you’ll find that the checkboxes for auto-opening this kind of windows are also gone. Don’t panic, these functions are just moved to the window manager. From now all of  your opened windows will be saved at exit and will be restored the same order at the next start of DC++.  All type of windows will be saved not just the ones that can be toggled on/off. You’ll find that your opened filelists and PM windows will also be kept across sessions. And yes, its the same for the opened hub windows as well.

WindowManager also stores the list of recently opened PM, filelist and hub windows .  The history of opened windows can be accessed through the dropdown besides the Recent Windows icon in the toolbar. You can reopen any recent window by selecting it from the dropdown menu. You can also control the behaviour of the Recent Windows function by specifying how many recently opened hubs, PM and file list windows you want to be saved. The number of stored recent items can be set in the newly introduced History panel in the Settings. Note that setting any value to zero means that the saving of the recent items for the respective type of windows will be disabled.

The second notable GUI functionality change happened at favorite hubs configuration. If you open the Favorite Hubs window you’ll see that the auto-connect checkboxes are disappeared from the list of hubs. This is because we introduced Favorite hub groups, a new and better way of managing your favorite places in the DC network. You can create unlimited number of hub groups using the brand new hub group dialog. Just click to Manage Groups button/context menu item in the Favorite hubs window and you can add/modify hub groups there.

The list style in the Favorite Hubs window is also changed to a grouped list view where you can see your hubs  grouped according the hub groups settings. Note that this grouped list view is available on Windows XP or newer operating systems only. In OSes older than XP,  the Hub Group column will tell you what group a favorite hub entry belongs to.

When you upgrade DC++, existing favorite hubs will be a member of  an unnamed default hub group. You can assign a favorite hub to any other group in the properties dialog of the favorite hub. You can also use the Move Up/Down buttons to move a hub in the grouped list from one group to another.

From now you can quick connect to all hubs in  a particular group using the appropriate submenu from the File menu or by simply pressing Ctrl-G. There’s also a quick way to close all opened members of a hub group or close all other hubs except the ones that are members of a particular group. You can find these functions under the Window menu.

Hub groups can be set so their members will be automatically connected when DC++ started.  When you upgrade from an older version of DC++, previously auto-connecting favorite hubs will be converted to a new hub group which has the auto-connect property set.  This way even those who aren’t suppose to use hub groups at all won’t experience any inconvenience.

Now you’re familiar with the two most important new features but DC++ 0.76x has many more smaller GUI and lib improvements. I’ll expose them in the following posts shortly…

DC++ 0.761

A new quick service release of DC++ has just out. It contains only stability fixes, so don’t expect any new functions or GUI changes this time. Upgrade is strongly recommended for everyone who is already on 0.760.

Design a site like this with WordPress.com
Get started