-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathnetworking.src
More file actions
108 lines (88 loc) · 5.21 KB
/
networking.src
File metadata and controls
108 lines (88 loc) · 5.21 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
%include('html_intro.inc')
<meta name="keywords" content="projects">
<meta name="description" content="NCCS TechInt group thrust area">
<title>Technology Integration Group: Networking</title>
%include('hdbody.inc')
%include('hdrnav.inc')
<!-- ========================================================== -->
%include('content_intro.inc')
<div class="overview">
The Technology Integration Group's Networking projects focus on data
movement at all scales and levels of abstraction -- among processors
within a supercomputer, among computers across continents, across
diverse hardware transport mechanisms using a variety of protocol
stacks.
</div>
<div class="plinks">
<a href="#CCI">Common Communication Interface</a>
<br><a href="#Terabit">Terabit Data Movement Optimization</a>
<br><a href="#Staged-Data-Movement">Staged Data Movement using Multiple Network Paths for Improved Throughput</a>
</div>
%include('content_close.inc')
<!-- ========================================================== -->
%include('content_intro.inc')
<div class="logo-flow">
<a
href="http://www.olcf.ornl.gov/center-projects/common-communication-interface/"
target="other">
<img class="networking"
src="https://www.olcf.ornl.gov/wp-content/uploads/2011/06/cci-with-text-590x289.png"
alt="CCI Logo"/>
</a>
</div>
<h2 id="CCI">Common Communication Interface (CCI)</h2>
<p>The CCI project is an open-source communication interface that aims to
provide a simple and portable API, high performance, scalability for
the largest deployments, and robustness in the presence of faults. It
is developed and maintained by a partnership of research, academic,
and industry members.</p>
<p>Targeted towards high performance computing (HPC) environments as well
as large data centers, CCI can provide a common network abstraction
layer (NAL) for persistent services as well as general interprocess
communication. In HPC, MPI is the de facto standard for communication
within a job. Persistent services such as distributed file systems,
code coupling (e.g. a simulation sending output to an analysis
application sending its output to a visualization process), health
monitoring, debugging, and performance monitoring, however, exist
outside of scheduler jobs or span multiple jobs. In these cases, these
persistent services tend to use either BSD sockets for portability to
avoid having to rewrite the applications for each new interconnect or
they implement their own NAL which takes developer time and effort.
CCI can simplify support for these persistent services by providing a
common NAL which minimizes the maintenance and support for these
services while providing improved performance (i.e. reduced latency
and increased bandwidth) compared to Sockets.</p>
<p>We are currently supporting Sockets (UDP and TCP), OpenFabrics Verbs
(InfiniBand and RoCE), Cray GNI (Gemini and Aries), Cray Portals 3.3
(SeaStar), D-Star (a native CCI implementation on SeaStar), as well as raw
Ethernet (IP-bypass). Visit <a href="http://cci-forum.com/"
target="other">CCI Forum</a> for release information.</p>
<p>Contributors: Scott Atchley
<p>Status: On-going
<p> <a class="right" href="#">Top</a>
%include('content_close.inc')
<!-- ========================================================== -->
%include('content_intro.inc')
<h2 id="Terabit">An Adaptive End-to-End Approach for Terabit Data Movement Optimization</h2>
<p>This is the work related to <em>An Adaptive End-to-End Approach for
Terabit Data Movement Optimization</em> project, the goal of this project
is to improve throughput of wide-area data transfer between DOE facilities.
This work involves serveral areas:</p>
<ul>
<li>Optimization of CCI for wide-area networking</li>
<li>Heterogenous routing of CCI</li>
<li>Optimization of accesses to parallel file systems</li>
</ul>
<p>Contributors: Scott Atchley, Youngjae Kim
<p>Status: Completed
<p> <a class="right" href="#">Top</a>
%include('content_close.inc')
<!-- ========================================================== -->
%include('content_intro.inc')
<h2 id="Staged-Data-Movement">Staged Data Movement using Multiple Network Paths for Improved Throughput</h2>
<p>This effort is working to develop tools that will improve bulk data movement of large scientific datasets between DOE's facilities and collaborating institutions. The initial tool will allow a user to specify a set of files, the source and destination institutions, as well as intermediate facilities that can assist in the transfer. The tool will then try to transfer some data directly from source to destination while simultaneously transferring data to the intermediate locations and from there to the destination. The user may see improved throughput by taking advantage of dedicated paths between the various locations. This is a form of overlay routing using common tools such as scp, bbcp, GridFTP, etc.</p>
<p>Contributors: Scott Atchley and Sudharshan Vazhkudai
<p>Status: Completed
<p> <a class="right" href="#">Top</a>
%include('content_close.inc')
%include('html_close.inc')