Skip to content

Commit b93ede6

Browse files
author
joseramon.afonso
committed
Added examples and fix batch_mode
1 parent 343cc60 commit b93ede6

File tree

9 files changed

+119
-22
lines changed

9 files changed

+119
-22
lines changed

‎devoutils/faker/generators/batch_fake_generator.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -83,7 +83,7 @@ def run(self):
8383

8484
counter = 0
8585
date_generator = self.date_range(
86-
self._start_date, self._end_date, self.frequency, self.probability,
86+
self._start_date, self._end_date, self.freq, self.prob,
8787
self.date_format, self.dont_remove_microseconds)
8888
with open(self.file_name, "w") as f:
8989
while True:

‎devoutils/faker/scripts/faker_cli.py‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -33,10 +33,6 @@ def cli():
3333
@click.option('--address', help='address to send.')
3434
@click.option('--port', help='Port to send.')
3535
@click.option('--tag', help='Tag from Devo.')
36-
@click.option('--file_name', help='File name to store events. If file name '
37-
'exist will only store the events in a file. Can be used '
38-
'with batch mode to set the file where store the batch '
39-
'events')
4036
@click.option('--simulation', is_flag=True,
4137
help='Set as simulation. Shows the event in the console, '
4238
'but do not send it')
@@ -48,6 +44,11 @@ def cli():
4844
help='Interactive mode.')
4945
@click.option('--raw_mode', '-raw', is_flag=True,
5046
help='Send raw mode.')
47+
@click.option('--file_name', help='Use File Fake Generator.'
48+
'File name to store events. If file name '
49+
'exist will only store the events in a file. Can be used '
50+
'with batch mode to set the file where store the batch '
51+
'events')
5152
@click.option('--probability', default=100, help='Probability (0-100).')
5253
@click.option('--frequency', default="1-1", help='Frequency in seconds. '
5354
'Example: '
@@ -91,13 +92,13 @@ def cli(**kwargs):
9192
spec.loader.exec_module(module)
9293
get_providers = getattr(module, provcfg.get("getter"))
9394
providers = get_providers()
95+
cfg.__delitem__("providers")
9496
except Exception as error:
9597
print_error("Error when loading Providers", show_help=False,
9698
stop=False)
9799
print_error(error, show_help=False, stop=True)
98100

99101
params = []
100-
101102
click.echo("» Press Ctrl+C to stop the process «", file=sys.stderr)
102103
if cfg['interactive']:
103104
click.echo("» Interactive mode «")

‎docs/faker/batch.md‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,13 @@ to generate much more data from the original sample.
1212

1313
The batch mode allows to generate events between two arbitrary dates as fast
1414
as possible. The events will be dumped to the standard output and then you can
15-
redirect it to a file or somewhere else
15+
redirect it to a file or somewhere else.
16+
17+
18+
You can generate raw or normal data, and when you have the file send with devo-sdk
19+
using the normal mode, or the raw mode. For this type of documents, when you want
20+
to generate much amount of data, the mor usefull is generate raw data and send it with
21+
raw mode in devo-sdk
1622

1723
First we want to take the sample events and create a template from them. For
1824
example, if we have these sample events:
@@ -71,3 +77,22 @@ Reusing the eventdate in other parts of the log:
7177
{%- set eventdate = next(date_generator) -%}
7278
{{ eventdate }} {{ fake.ipv4() }} WEndpoint_Profile 1234567890 1 0 mac_address=10acbdefg01,ip_address={{ fake.ipv4() }},static_ip=t,hostname=DEVOHOSTNAME,username={{ 'USER%s' %fake.int(0, 9) }},login_status={{ fake.random_element({'ACCEPT': 0.4, 'REJECT': 0.1, 'TIMEOUT': 0.8}) }},error_code={{ fake.random_element((0, 106)) }}
7379
```
80+
81+
### CLI Usage
82+
83+
You can use this generator with the option `--batch_mode`
84+
85+
86+
devo-faker --template "~/python-utils/examples/simple_cli_example.jinja2" --config "config.yaml" --batch_mode
87+
88+
config.yaml:
89+
90+
faker:
91+
probability: 80
92+
frequency: (2,5)
93+
date_range: ("2020-01-01 00:00:00", "2020-01-02 00:00:00")
94+
file_name: "example_batch.txt"
95+
96+
97+
98+
You have more info, flags and options in [Terminal/Shell CLI usage](shellcli.md)

‎docs/faker/file.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,3 +43,14 @@ Template:
4343
{%- set type = fake.random_element(["post", "get"]) -%}
4444
{{ next(date_generator) }} receiving {{ type }} request.
4545
{%- set a = 3 -%}
46+
47+
### CLI Usage
48+
49+
You can use this generator with the option `--file_name 'name_if_file.txt`
50+
51+
52+
devo-faker --template "~/python-utils/examples/simple_cli_example.jinja2" --config "config.yaml" --file_name "example.txt"
53+
54+
This `--file_name` if is used with flag `--batch_mode` is used in BatchFakeGenerator
55+
56+
You have more info, flags and options in [Terminal/Shell CLI usage](shellcli.md)

‎docs/faker/shellcli.md‎

Lines changed: 25 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -28,18 +28,18 @@ Options:
2828
--address TEXT address to send.
2929
--port TEXT Port to send.
3030
--tag TEXT Tag from Devo.
31-
--file_name TEXT File name to store events. If file name exist
32-
will only store the events in a file. Can be
33-
used with batch mode to set the file where
34-
store the batch events
35-
3631
--simulation Set as simulation. Shows the event in the
3732
console, but do not send it
3833
3934
-t, --template FILENAME Template to send. [required]
4035
-p, --providers FILENAME File with custom providers dict.
4136
-i, --interactive Interactive mode.
4237
-raw, --raw_mode Send raw mode.
38+
--file_name TEXT Use File Fake Generator.File name to store
39+
events. If file name exist will only store the
40+
events in a file. Can be used with batch mode
41+
to set the file where store the batch events
42+
4343
--probability INTEGER Probability (0-100).
4444
--frequency TEXT Frequency in seconds. Example: "1.0-5.0" =
4545
random time between 1 sec. to 5secs.
@@ -65,6 +65,13 @@ Options:
6565
--help Show this message and exit.
6666
```
6767

68+
69+
You can use all the flags/values in a config file, using the name of the flags with two "-"
70+
For example, for `-raw, --raw_mode` you need use in config file the value `raw_mode`
71+
for `-i, --interactive`, `interactive`, etc.
72+
73+
And all values into one object called "faker" in the main level:
74+
6875
The config.json file could be like this:
6976
```
7077
{
@@ -76,9 +83,13 @@ The config.json file could be like this:
7683
"address": "eu.elb.relay.logtrust.net",
7784
"port": 443
7885
},
79-
"verbose": true,
80-
"probability": 80,
81-
"frequency": (2,4)
86+
"faker"{
87+
"verbose": true,
88+
"probability": 80,
89+
"frequency": (2,4),
90+
"interactive": true,
91+
"template": "/path/to/template.file"
92+
}
8293
}
8394
```
8495

@@ -91,8 +102,10 @@ sender:
91102
tag: "test.keep.free",
92103
address: "eu.elb.relay.logtrust.net",
93104
port: 443
94-
simulation: true
95-
probability: 50
96-
frequency: (1,3)
97-
providers: providers.py
105+
faker:
106+
simulation: true
107+
probability: 50
108+
batch_mode: true
109+
dont_remove_microseconds: true
110+
providers: providers.py
98111
```

‎docs/faker/simulation.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,12 @@ would work, etc., without risking sending data.
1111

1212
In short, it is a generator to display the data on the screen, as if you activated the flag
1313
`simulation` and` verbose` in any of the other generators
14+
15+
### CLI Usage
16+
17+
If you take any use of the CLI, be it file, bash, realtime, etc.,
18+
and use the `--simulation` flag, instead of executing the chosen FakeGenerator,
19+
simulation will be used, which does not send or write anywhere, but displays in a screen
20+
the result of the template and the configuration values
21+
22+
You have more info, flags and options in [Terminal/Shell CLI usage](shellcli.md)

‎docs/faker/syslog.md‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,26 @@ Template:
4343
{%- set type = fake.random_element(["post", "get"]) -%}
4444
{{ next(date_generator) }} receiving {{ type }} request.
4545
{%- set a = 3 -%}
46+
47+
48+
### CLI Usage
49+
50+
You can use this generator without any flags. Its the default execution of cli shell
51+
52+
53+
devo-faker --template "~/python-utils/examples/simple_syslog_template.jinja2" --config "config.yaml"
54+
55+
config.yaml:
56+
57+
faker:
58+
probability: 80
59+
frequency: (2,5)
60+
sender:
61+
*devo-sdk configuration
62+
63+
64+
You have more info, flags and options in [Terminal/Shell CLI usage](shellcli.md)
65+
4666

4767

4868
## Raw syslog fake generator
@@ -61,3 +81,22 @@ create the data, in your template you will need to generate the entire line to s
6181
This generator does not need the tag, since you add it in the headers
6282

6383
<14>Jan 1 00:00:00 MacBook-Pro-de-X.local my.app.devo_sender.test: my data
84+
85+
86+
### CLI Usage
87+
88+
You can use this generator with the option `--raw_mode`
89+
90+
91+
devo-faker --template "~/python-utils/examples/simple_syslog_template.jinja2" --config "config.yaml" --raw_mode
92+
93+
config.yaml:
94+
95+
faker:
96+
probability: 80
97+
frequency: (2,5)
98+
sender:
99+
*devo-sdk configuration
100+
101+
102+
You have more info, flags and options in [Terminal/Shell CLI usage](shellcli.md)

‎requirements.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
click==7.1.1
22
requests==2.23.0
33
PyYAML==5.3.1
4-
devo-sdk==3.3.0
4+
devo-sdk==3.3.2
55
Jinja2==2.11.1
66
psutil==5.7.0
77
python-dateutil==2.8.1

‎setup.py‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,11 +24,10 @@
2424
"Topic :: Software Development :: Libraries :: Python Modules",
2525
]
2626

27-
INSTALL_REQUIRES = ['devo-sdk==3.3.0',
27+
INSTALL_REQUIRES = ['devo-sdk==3.3.2',
2828
'click==7.1.1',
2929
'requests==2.23.0',
3030
'PyYAML==5.3.1',
31-
'devo-sdk==3.3.0',
3231
'Jinja2==2.11.1',
3332
'psutil==5.7.0',
3433
'python-dateutil==2.8.1',

0 commit comments

Comments
 (0)