Skip to content

Commit f689d5d

Browse files
eng-waleed1eng-waleed1
authored
docs-issue-OpenHands#2887: Create openshift-example.md (OpenHands#2960)
docs-issue-OpenHands#2887: Create openshift-example.md
1 parent f80ecec commit f689d5d

1 file changed

Lines changed: 302 additions & 0 deletions

File tree

‎docs/modules/usage/openshift-example.md‎

Lines changed: 302 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,302 @@
1+
---
2+
sidebar_position: 6
3+
---
4+
5+
# 💿 How to use OpenDevin in OpenShift/K8S
6+
7+
There are different ways and scenarios that you can do, we're just mentioning one example here:
8+
1. Create a PV "as a cluster admin" to map workspace_base data and docker directory to the pod through the worker node.
9+
2. Create a PVC to be able to mount those PVs to the POD
10+
3. Create a POD which contains two containers; the OpenDevin and Sandbox containers.
11+
12+
## Steps to follow the above example.
13+
14+
> Note: Make sure you are logged in to the cluster first with the proper account for each step. PV creation requires cluster administrator!
15+
16+
> Make sure you have read/write permissions on the hostPath used below (i.e. /tmp/workspace)
17+
18+
1. Create the PV:
19+
Sample yaml file below can be used by a cluster admin to create the PV.
20+
- workspace-pv.yaml
21+
22+
```yamlfile
23+
apiVersion: v1
24+
kind: PersistentVolume
25+
metadata:
26+
name: workspace-pv
27+
spec:
28+
capacity:
29+
storage: 2Gi
30+
accessModes:
31+
- ReadWriteOnce
32+
persistentVolumeReclaimPolicy: Retain
33+
hostPath:
34+
path: /tmp/workspace
35+
```
36+
37+
```bash
38+
# apply yaml file
39+
$ oc create -f workspace-pv.yaml
40+
persistentvolume/workspace-pv created
41+
42+
# review:
43+
$ oc get pv
44+
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
45+
workspace-pv 2Gi RWO Retain Available 7m23s
46+
```
47+
48+
- docker-pv.yaml
49+
50+
```yamlfile
51+
apiVersion: v1
52+
kind: PersistentVolume
53+
metadata:
54+
name: docker-pv
55+
spec:
56+
capacity:
57+
storage: 2Gi
58+
accessModes:
59+
- ReadWriteOnce
60+
persistentVolumeReclaimPolicy: Retain
61+
hostPath:
62+
path: /var/run/docker.sock
63+
```
64+
65+
```bash
66+
# apply yaml file
67+
$ oc create -f docker-pv.yaml
68+
persistentvolume/docker-pv created
69+
70+
# review:
71+
oc get pv
72+
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
73+
docker-pv 2Gi RWO Retain Available 6m55s
74+
workspace-pv 2Gi RWO Retain Available 7m23s
75+
```
76+
77+
2. Create the PVC:
78+
Sample PVC yaml file below:
79+
80+
- workspace-pvc.yaml
81+
82+
```yamlfile
83+
apiVersion: v1
84+
kind: PersistentVolumeClaim
85+
metadata:
86+
name: workspace-pvc
87+
spec:
88+
accessModes:
89+
- ReadWriteOnce
90+
resources:
91+
requests:
92+
storage: 1Gi
93+
```
94+
95+
```bash
96+
# create the pvc
97+
$ oc create -f workspace-pvc.yaml
98+
persistentvolumeclaim/workspace-pvc created
99+
100+
# review
101+
$ oc get pvc
102+
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
103+
workspace-pvc Pending hcloud-volumes 4s
104+
105+
$ oc get events
106+
LAST SEEN TYPE REASON OBJECT MESSAGE
107+
8s Normal WaitForFirstConsumer persistentvolumeclaim/workspace-pvc waiting for first consumer to be created before binding
108+
```
109+
110+
- docker-pvc.yaml
111+
112+
```yamlfile
113+
apiVersion: v1
114+
kind: PersistentVolumeClaim
115+
metadata:
116+
name: docker-pvc
117+
spec:
118+
accessModes:
119+
- ReadWriteOnce
120+
resources:
121+
requests:
122+
storage: 1Gi
123+
```
124+
125+
```bash
126+
# create pvc
127+
$ oc create -f docker-pvc.yaml
128+
persistentvolumeclaim/docker-pvc created
129+
130+
# review
131+
$ oc get pvc
132+
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
133+
docker-pvc Pending hcloud-volumes 4s
134+
workspace-pvc Pending hcloud-volumes 2m53s
135+
136+
$ oc get events
137+
LAST SEEN TYPE REASON OBJECT MESSAGE
138+
10s Normal WaitForFirstConsumer persistentvolumeclaim/docker-pvc waiting for first consumer to be created before binding
139+
10s Normal WaitForFirstConsumer persistentvolumeclaim/workspace-pvc waiting for first consumer to be created before binding
140+
```
141+
142+
3. Create the POD yaml file:
143+
Sample POD yaml file below:
144+
145+
- pod.yaml
146+
147+
```yamlfile
148+
apiVersion: v1
149+
kind: Pod
150+
metadata:
151+
name: opendevin-app-2024
152+
labels:
153+
app: opendevin-app-2024
154+
spec:
155+
containers:
156+
- name: opendevin-app-2024
157+
image: ghcr.io/opendevin/opendevin:0.7.1
158+
env:
159+
- name: SANDBOX_USER_ID
160+
value: "1000"
161+
- name: SANDBOX_BOX_TYPE
162+
value: 'local'
163+
- name: WORKSPACE_MOUNT_PATH
164+
value: "/opt/workspace_base"
165+
volumeMounts:
166+
- name: workspace-volume
167+
mountPath: /opt/workspace_base
168+
- name: docker-sock
169+
mountPath: /var/run/docker.sock
170+
ports:
171+
- containerPort: 3000
172+
- name: opendevin-sandbox-2024
173+
image: ghcr.io/opendevin/sandbox:main
174+
ports:
175+
- containerPort: 51963
176+
command: ["/usr/sbin/sshd", "-D", "-p 51963", "-o", "PermitRootLogin=yes"]
177+
volumes:
178+
- name: workspace-volume
179+
persistentVolumeClaim:
180+
claimName: workspace-pvc
181+
- name: docker-sock
182+
persistentVolumeClaim:
183+
claimName: docker-pvc
184+
```
185+
186+
```bash
187+
# create the pod
188+
$ oc create -f pod.yaml
189+
W0716 11:22:07.776271 107626 warnings.go:70] would violate PodSecurity "restricted:v1.24": allowPrivilegeEscalation != false (containers "opendevin-app-2024", "opendevin-sandbox-2024" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (containers "opendevin-app-2024", "opendevin-sandbox-2024" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or containers "opendevin-app-2024", "opendevin-sandbox-2024" must set securityContext.runAsNonRoot=true), seccompProfile (pod or containers "opendevin-app-2024", "opendevin-sandbox-2024" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
190+
pod/opendevin-app-2024 created
191+
192+
# Above warning can be ignored for now as we will not modify SCC restrictions.
193+
194+
# review
195+
$ oc get pods
196+
NAME READY STATUS RESTARTS AGE
197+
opendevin-app-2024 0/2 Pending 0 5s
198+
199+
$ oc get pods
200+
NAME READY STATUS RESTARTS AGE
201+
opendevin-app-2024 0/2 ContainerCreating 0 15s
202+
203+
$ oc get events
204+
LAST SEEN TYPE REASON OBJECT MESSAGE
205+
38s Normal WaitForFirstConsumer persistentvolumeclaim/docker-pvc waiting for first consumer to be created before binding
206+
23s Normal ExternalProvisioning persistentvolumeclaim/docker-pvc waiting for a volume to be created, either by external provisioner "csi.hetzner.cloud" or manually created by system administrator
207+
27s Normal Provisioning persistentvolumeclaim/docker-pvc External provisioner is provisioning volume for claim "opendevin/docker-pvc"
208+
17s Normal ProvisioningSucceeded persistentvolumeclaim/docker-pvc Successfully provisioned volume pvc-2b1d223a-1c8f-4990-8e3d-68061a9ae252
209+
16s Normal Scheduled pod/opendevin-app-2024 Successfully assigned opendevin/opendevin-app-2024 to worker1.hub.internal.blakane.com
210+
9s Normal SuccessfulAttachVolume pod/opendevin-app-2024 AttachVolume.Attach succeeded for volume "pvc-2b1d223a-1c8f-4990-8e3d-68061a9ae252"
211+
9s Normal SuccessfulAttachVolume pod/opendevin-app-2024 AttachVolume.Attach succeeded for volume "pvc-31f15b25-faad-4665-a25f-201a530379af"
212+
6s Normal AddedInterface pod/opendevin-app-2024 Add eth0 [10.128.2.48/23] from openshift-sdn
213+
6s Normal Pulled pod/opendevin-app-2024 Container image "ghcr.io/opendevin/opendevin:0.7.1" already present on machine
214+
6s Normal Created pod/opendevin-app-2024 Created container opendevin-app-2024
215+
6s Normal Started pod/opendevin-app-2024 Started container opendevin-app-2024
216+
6s Normal Pulled pod/opendevin-app-2024 Container image "ghcr.io/opendevin/sandbox:main" already present on machine
217+
5s Normal Created pod/opendevin-app-2024 Created container opendevin-sandbox-2024
218+
5s Normal Started pod/opendevin-app-2024 Started container opendevin-sandbox-2024
219+
83s Normal WaitForFirstConsumer persistentvolumeclaim/workspace-pvc waiting for first consumer to be created before binding
220+
27s Normal Provisioning persistentvolumeclaim/workspace-pvc External provisioner is provisioning volume for claim "opendevin/workspace-pvc"
221+
17s Normal ProvisioningSucceeded persistentvolumeclaim/workspace-pvc Successfully provisioned volume pvc-31f15b25-faad-4665-a25f-201a530379af
222+
223+
$ oc get pods
224+
NAME READY STATUS RESTARTS AGE
225+
opendevin-app-2024 2/2 Running 0 23s
226+
227+
$ oc get pvc
228+
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
229+
docker-pvc Bound pvc-2b1d223a-1c8f-4990-8e3d-68061a9ae252 10Gi RWO hcloud-volumes 10m
230+
workspace-pvc Bound pvc-31f15b25-faad-4665-a25f-201a530379af 10Gi RWO hcloud-volumes 13m
231+
232+
```
233+
234+
4. Create a NodePort service.
235+
Sample service creation command below:
236+
237+
```bash
238+
# create the service of type NodePort
239+
$ oc create svc nodeport opendevin-app-2024 --tcp=3000:3000
240+
service/opendevin-app-2024 created
241+
242+
# review
243+
244+
$ oc get svc
245+
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
246+
opendevin-app-2024 NodePort 172.30.225.42 <none> 3000:30495/TCP 4s
247+
248+
$ oc describe svc opendevin-app-2024
249+
Name: opendevin-app-2024
250+
Namespace: opendevin
251+
Labels: app=opendevin-app-2024
252+
Annotations: <none>
253+
Selector: app=opendevin-app-2024
254+
Type: NodePort
255+
IP Family Policy: SingleStack
256+
IP Families: IPv4
257+
IP: 172.30.225.42
258+
IPs: 172.30.225.42
259+
Port: 3000-3000 3000/TCP
260+
TargetPort: 3000/TCP
261+
NodePort: 3000-3000 30495/TCP
262+
Endpoints: 10.128.2.48:3000
263+
Session Affinity: None
264+
External Traffic Policy: Cluster
265+
Events: <none>
266+
```
267+
268+
6. Connect to OpenDevin UI, configure the Agent, then test:
269+
270+
![image](https://github.com/user-attachments/assets/12f94804-a0c7-4744-b873-e003c9caf40e)
271+
272+
273+
## Challenges
274+
Some of the challenages that would be needed to improve:
275+
276+
1. Install GIT into the container:
277+
This can be resolved by building a custom image which includes GIT software and use that image during pod deplyment.
278+
279+
Example below: "to be tested!"
280+
281+
```dockerfile
282+
FROM ghcr.io/opendevin/opendevin:0.7.1
283+
284+
# Install Git
285+
RUN apt-get update && apt-get install -y git
286+
287+
# Ensure /opt/workspace_base is writable
288+
RUN mkdir -p /opt/workspace_base && chown -R 1000:1000 /opt/workspace_base
289+
290+
# Verify Git installation
291+
RUN git --version
292+
```
293+
294+
2. Mount a shared development directory "i.e. one hosted in EC2 instance" to the POD:
295+
This can be also done by sharing the developement directory to the worker node through a sharing software (NFS), then creating a pv and pvc as described above to access that directory.
296+
297+
3. Not all Agents working! Just tested CoderAgent with an openai API key and produced results.
298+
299+
300+
## Discuss
301+
302+
For other issues or questions join the [Slack](https://join.slack.com/t/opendevin/shared_invite/zt-2jsrl32uf-fTeeFjNyNYxqSZt5NPY3fA) or [Discord](https://discord.gg/ESHStjSjD4) and ask!

0 commit comments

Comments
 (0)