Skip to content

Commit 8572dde

Browse files
authored
Setup Travis CI (ibm-functions#8)
* Setup Travis CI
1 parent 320211f commit 8572dde

29 files changed

Lines changed: 1150 additions & 525 deletions

.gitignore

Lines changed: 27 additions & 98 deletions
Original file line numberDiff line numberDiff line change
@@ -1,103 +1,32 @@
1-
# Byte-compiled / optimized / DLL files
2-
__pycache__/
3-
*.py[cod]
4-
*$py.class
5-
6-
# C extensions
7-
*.so
8-
9-
# Distribution / packaging
10-
.Python
11-
env/
1+
# Tests
2+
tests/.cache-main
3+
tests/.classpath
4+
tests/.gitignore
5+
tests/.project
6+
tests/.settings/
7+
tests/credentials.json
8+
tests/vcap_services.json
9+
10+
# Linux
11+
*~
12+
13+
# Mac
14+
.DS_Store
15+
16+
# Gradle
17+
.gradle
1218
build/
13-
develop-eggs/
14-
dist/
15-
downloads/
16-
eggs/
17-
.eggs/
18-
lib/
19-
lib64/
20-
parts/
21-
sdist/
22-
var/
23-
wheels/
24-
*.egg-info/
25-
.installed.cfg
26-
*.egg
27-
28-
# PyInstaller
29-
# Usually these files are written by a python script from a template
30-
# before PyInstaller builds the exe, so as to inject date/other infos into it.
31-
*.manifest
32-
*.spec
33-
34-
# Installer logs
35-
pip-log.txt
36-
pip-delete-this-directory.txt
37-
38-
# Unit test / coverage reports
39-
htmlcov/
40-
.tox/
41-
.coverage
42-
.coverage.*
43-
.cache
44-
nosetests.xml
45-
coverage.xml
46-
*.cover
47-
.hypothesis/
48-
49-
# Translations
50-
*.mo
51-
*.pot
52-
53-
# Django stuff:
54-
*.log
55-
local_settings.py
56-
57-
# Flask stuff:
58-
instance/
59-
.webassets-cache
60-
61-
# Scrapy stuff:
62-
.scrapy
63-
64-
# Sphinx documentation
65-
docs/_build/
66-
67-
# PyBuilder
68-
target/
69-
70-
# Jupyter Notebook
71-
.ipynb_checkpoints
72-
73-
# pyenv
74-
.python-version
75-
76-
# celery beat schedule file
77-
celerybeat-schedule
78-
79-
# SageMath parsed files
80-
*.sage.py
81-
82-
# dotenv
83-
.env
84-
85-
# virtualenv
86-
.venv
87-
venv/
88-
ENV/
89-
90-
# Spyder project settings
91-
.spyderproject
92-
.spyproject
19+
!/tools/build/
9320

94-
# Rope project settings
95-
.ropeproject
21+
# IntelliJ
22+
.idea
23+
*.class
24+
out/
25+
*.iml
9626

97-
# mkdocs documentation
98-
/site
27+
# VSCode
28+
.vscode
9929

100-
# mypy
101-
.mypy_cache/
30+
#Python
31+
virtualenv/
10232

103-
.gradle/**

.scalafmt.conf

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
style = intellij
2+
danglingParentheses = false
3+
maxColumn = 120
4+
docstrings = JavaDoc
5+
rewrite.rules = [SortImports]
6+
project.git = true

.travis.yml

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
sudo: required
2+
group: deprecated-2017Q3
3+
language: scala
4+
scala:
5+
- 2.11.8
6+
services:
7+
- docker
8+
before_install:
9+
- "./tools/travis/setup.sh"
10+
install: true
11+
script:
12+
- "./tools/travis/build.sh && ./tools/travis/deploy.sh && ./tools/travis/test.sh"
13+
deploy:
14+
- provider: script
15+
script: "./tools/travis/publish.sh ibmfunctions latest"
16+
on:
17+
branch: master
18+
repo: ibm-functions/runtime-python
19+
- provider: script
20+
script: "./tools/travis/publish.sh ibmfunctions $TRAVIS_TAG"
21+
on:
22+
tags: true
23+
all_branches: true
24+
repo: ibm-functions/runtime-python

README.md

Lines changed: 78 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,80 @@
1-
# local development
1+
# IBM Cloud Functions runtime for python
2+
3+
WARNING: Work in Progress (WIP) not ready for production
4+
5+
[![Build Status](https://travis-ci.org/ibm-functions/runtime-python.svg?branch=master)](https://travis-ci.org/ibm-functions/runtime-python)
6+
7+
The runtime provides [python v3](python3/) with a set of [python packages](python/requirements.txt)
8+
9+
The runtime provides the following npm packages for [IBM Cloud](https://bluemix.net):
10+
- IBM DB2/DashDB and IBM Informix [[email protected]](https://pypi.python.org/pypi/ibm_db)
11+
- IBM Cloudant [[email protected]](https://pypi.python.org/pypi/cloudant)
12+
- IBM Watson Developer Cloud Python SDK [[email protected]](https://pypi.python.org/pypi/watson-developer-cloud)
13+
14+
### How to use as a docker Action
15+
To use as a docker action
16+
```
17+
bx wsk action update myAction myAction.js --docker ibmfunctions/action-python-v3
18+
```
19+
This works on any deployment of Apache OpenWhisk or IBM Cloud Functions
20+
21+
### Future: IBM Cloud Functions (based on Apache OpenWhisk)
22+
To use as a python kind action
23+
```
24+
bx wsk action update myAction myAction --kind python:3
25+
```
26+
Tip: Not available yet in the IBM Cloud
27+
28+
### Working with the local git repo
29+
Prerequisite: *Export* OPENWHISK_HOME to point to your incubator/openwhisk cloned directory.
30+
31+
```
32+
./gradlew nodejs8:distDocker
33+
```
34+
This will produce the image `whisk/action-python-v3`
35+
36+
Build and Push image
237
```
3-
./gradlew python3:distDocker
438
docker login
5-
./gradlew python3:distDocker -PdockerImagePrefix=<YourDockerId> -PdockerRegistry=docker.io
6-
cd $OPENWHISK_HOME
7-
bin/wskdev teardown -t <currentEnvironmentName>
8-
cd ansible/environments/
9-
ln -s /path/to/runtime-python3/ansible/environments/local ./local-ibm-py
10-
cd ..
11-
ansible-playbook -i environments/local-ibm-py setup.yml
12-
ansible-playbook -i environments/local-ibm-py couchdb.yml -e mode=clean
13-
ansible-playbook -i environments/local-ibm-py couchdb.yml
14-
ansible-playbook -i environments/local-ibm-py initdb.yml
15-
ansible-playbook -i environments/local-ibm-py wipe.yml
16-
cd $OPENWHISK_HOME
17-
./gradlew distDocker
18-
cd ansible
19-
ansible-playbook -i environments/local-ibm-py openwhisk.yml -e mode=clean
20-
ansible-playbook -i environments/local-ibm-py openwhisk.yml
21-
ansible-playbook -i environments/local-ibm-py postdeploy.yml
22-
23-
```
24-
25-
Create a python action and put it in file `test.py`:
26-
```
27-
def main(args):
28-
return {"message":"Successfully called Simple Python Action"}
29-
```
30-
31-
Run:
32-
```
33-
wsk -i action create test test.py --kind "python-ibm:3"
34-
wsk -i action invoke test -b
35-
```
39+
./gradlew python3:distDocker -PdockerImagePrefix=$prefix-user -PdockerRegistry=docker.io
40+
```
41+
42+
Deploy OpenWhisk using ansible environment that adds the new king `python:3`
43+
Assuming you have OpenWhisk already deploy localy and `OPENWHISK_HOME` pointing to root directory of OpenWhisk core repository.
44+
45+
Set `ROOTDIR` to the root directory of this repository.
46+
47+
Redeploy OpenWhisk
48+
```
49+
cd $OPENWHISK_HOME/ansible
50+
ANSIBLE_CMD="ansible-playbook -i ${ROOTDIR}/ansible/environments/local"
51+
$ANSIBLE_CMD setup.yml
52+
$ANSIBLE_CMD couchdb.yml
53+
$ANSIBLE_CMD initdb.yml
54+
$ANSIBLE_CMD wipe.yml
55+
$ANSIBLE_CMD openwhisk.yml
56+
```
57+
58+
To use as docker action push to your own dockerhub account
59+
```
60+
docker tag whisk/action-python-v3 $user_prefix/action-python-v3
61+
docker push $user_prefix/action-python-v3
62+
```
63+
Then create the action using your the image from dockerhub
64+
```
65+
wsk action update myAction myAction.py --docker $user_prefix/action-python-v3
66+
```
67+
The `$user_prefix` is usually your dockerhub user id.
68+
69+
### Testing
70+
71+
72+
To run all tests: `./gradlew tests:test` this include tests depending on credentials
73+
74+
To run all tests except those which do not rely on credentials `./gradlew tests:testWithoutCredentials`
75+
76+
To run a single test-class: `./gradlew tests:test --tests <SomeGradleTestFilter>`
77+
78+
79+
# License
80+
[Apache 2.0](LICENSE.txt)

ansible/environments/local/group_vars/all

Lines changed: 3 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
whisk_version_name: local
2-
config_root_dir: /tmp
2+
config_root_dir: /tmp/wskconf
33
whisk_logs_dir: /tmp/wsklogs
44
docker_registry: ""
55
docker_dns: ""
@@ -22,8 +22,6 @@ apigw_host_v2: "http://{{ groups['apigateway']|first }}:{{apigateway.port.api}}/
2222
controller_arguments: '-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=1098'
2323
invoker_arguments: "{{ controller_arguments }}"
2424

25-
invoker_allow_multiple_instances: true
26-
2725
runtimes_manifest:
2826
defaultImagePrefix: "openwhisk"
2927
defaultImageTag: "latest"
@@ -35,23 +33,15 @@ runtimes_manifest:
3533
name: "nodejs6action"
3634
deprecated: false
3735
python:
38-
- kind: "python"
39-
image:
40-
name: "python2action"
41-
deprecated: false
4236
- kind: "python:2"
4337
default: true
4438
image:
4539
name: "python2action"
4640
deprecated: false
47-
- kind: "python:3"
48-
image:
49-
name: "python3action"
50-
deprecated: false
51-
- kind: "python-ibm:3"
41+
- kind: "python-jessie:3"
5242
default: false
5343
image:
54-
name: "action-python-ibm-3"
44+
name: "action-python-v3"
5545
deprecated: false
5646
blackboxes:
5747
- name: "dockerskeleton"

ansible/environments/local/hosts

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,11 @@ ansible ansible_connection=local
99
[controllers]
1010
controller0 ansible_host=172.17.0.1 ansible_connection=local
1111

12-
[kafka]
13-
172.17.0.1 ansible_host=172.17.0.1 ansible_connection=local
12+
[kafkas]
13+
kafka0 ansible_host=172.17.0.1 ansible_connection=local
14+
15+
[zookeepers:children]
16+
kafkas
1417

1518
[invokers]
1619
invoker0 ansible_host=172.17.0.1 ansible_connection=local

gradle/README.md

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Gradle
2+
3+
Gradle is used to build OpenWhisk. It does not need to be pre-installed as it installs itself using the [Gradle Wrapper](https://docs.gradle.org/current/userguide/gradle_wrapper.html). To use it without installing, simply invoke the `gradlew` command at the root of the repository. You can also install `gradle` via [`apt`](http://linuxg.net/how-to-install-gradle-2-1-on-ubuntu-14-10-ubuntu-14-04-ubuntu-12-04-and-derivatives/) on Ubuntu or [`brew`](http://www.brewformulas.org/Gradle) on Mac. In the following we use `gradle` and `gradlew` as synonymous.
4+
5+
## Usage
6+
7+
In general, project level properties are set via `-P{propertyName}={propertyValue}`. A task is called via `gradle {taskName}` and a subproject task is called via `gradle :path:to:subproject:{taskName}`. To run tasks in parallel, use the `--parallel` flag (**Note:** It's an incubating feature and might break stuff).
8+
9+
### Build
10+
11+
To build all Docker images use `gradle distDocker` at the top level project, to build a specific component use `gradle :core:controller:distDocker`.
12+
13+
Project level options that can be used on `distDocker`:
14+
15+
- `dockerImageName` (*required*): The name of the image to build (e.g. whisk/controller)
16+
- `dockerHost` (*optional*): The docker host to run commands on, default behaviour is docker's own `DOCKER_HOST` environment variable
17+
- `dockerRegistry` (*optional*): The registry to push to
18+
- `dockerImageTag` (*optional*, default 'latest'): The tag for the image
19+
- `dockerTimeout` (*optional*, default 240): Timeout for docker operations in seconds
20+
- `dockerRetries` (*optional*, default 3): How many times to retry docker operations
21+
- `dockerBinary` (*optional*, default `docker`): The binary to execute docker commands
22+
23+
### Test
24+
25+
To run tests one uses the `test` task. OpenWhisk consolidates tests into a single `tests` project. Hence the command to run all tests is `gradle :tests:test`.
26+
27+
It is possible to run specific tests using [Gradle testfilters](https://docs.gradle.org/current/userguide/java_plugin.html#test_filtering). For example `gradle :tests:test --tests "your.package.name.TestClass.evenMethodName"`. Wildcard `*` may be used anywhere.
28+
29+
## Build your own `build.gradle`
30+
In Gradle, most of the tasks we use are default tasks provided by plugins in Gradle. The [`scala` Plugin](https://docs.gradle.org/current/userguide/scala_plugin.html) for example includes tasks, that are needed to build Scala projects. Moreover, Gradle is aware of *Applications*. The [`application` Plugin](https://docs.gradle.org/current/userguide/application_plugin.html) provides tasks that are required to distribute a self-contained application. When `application` and `scala` are used in conjunction, they hook into each other and provide the tasks needed to distribute a Scala application. `distTar` for example compiles the Scala code, creates a jar containing the compiled classes and resources and creates a Tarball including that jar and all of its dependencies (defined in the dependencies section of `build.gradle`). It also creates a start-script which correctly sets the classpath for all those dependencies and starts the app.
31+
32+
In OpenWhisk, we want to distribute our application via Docker images. Hence we wrote a "plugin" that creates the task `distDocker`. That task will build an image from the `Dockerfile` that is located next to the `build.gradle` it is called from, for example Controller's `Dockerfile` and `build.gradle` are both located at `core/controller`.
33+
34+
If you want to create a new `build.gradle` for your component, simply put the `Dockerfile` right next to it and include `docker.gradle` by using
35+
36+
```
37+
ext.dockerImageName = 'openwwhisk/{IMAGENAME}'
38+
apply from: 'path/to/docker.gradle'
39+
```
40+
41+
If your component needs to be build before you can build the image, make `distDocker` depend on any task needed to run before it, for example:
42+
43+
```
44+
distDocker.dependsOn ':common:scala:distDocker', 'distTar'
45+
```

0 commit comments

Comments
 (0)