{"id":406,"date":"2025-03-18T21:49:31","date_gmt":"2025-03-18T20:49:31","guid":{"rendered":"https:\/\/pythonium.net\/?page_id=406"},"modified":"2025-03-18T21:49:00","modified_gmt":"2025-03-18T19:49:00","slug":"dataset","status":"publish","type":"page","link":"https:\/\/pythonium.net\/dataset","title":{"rendered":"Data maker"},"content":{"rendered":"\n<style>\n.deca {margin-left: 20px; margin-right: 20px;}\ntextarea {\n    height: 90px;\n    line-height: 100%;\n}\n.editor-valid {\n    color: green;\n    padding: 8px;\n    background-color: #d0f5a9;\n    margin: 10px 0;\n}\n.editor-error {\n    color: red;\n    padding: 8px;\n    background-color: #f6cece;\n    margin: 10px 0;\n}\n.editor-valid , .editor-error  {\n    width: 90%;\n    margin-right: auto;\n    margin-left: auto;\n}\n\n\n.content {\n    padding: 0px; \n    padding-top: 0px; \n    width: auto;\n }\n.page-content {\n    max-width: none; \n    width: 100%;\n}\n.single-post .post-header {\n    margin-bottom: 0px;\n}\ninput[type=\"text\"], select, input[type=\"number\"] {\npadding: 7px;\n}\n#templates-action input[type=\"button\"] {\nmargin-top: 15px;\ntext-align:center;\nwidth: 90%;\nmargin-left:auto;\nmargin-right:auto;\n}\n.current-schema {\ntext-align:center;\nmargin-top: 9px;\n}\n#container-result-files {\n    margin-top:20px;\n    margin: auto;\n    max-width: 600px;\n}\n.template-container {\nmargin-top: 9px;\n}\n.section-container {\npadding: 12px;\n}\n#list-templates {\npadding: 7px;\n}\n#list-templates a {\nmargin-top: 6px;\n}\n.container-center {\ntext-align: center;\n\n}\ninput[type=\"button\"].action-button-editor{\nfloat: right; margin-right: 40px; \n}\ninput[type=\"button\"].csv-add-field {\nfloat: none;\n}\n.section-title {\ntext-align: center;\nfont-size:1.4em;\nfont-weight:bold;\nmargin-bottom: 3px;\n}\nhr {\n    margin: 5px auto;\n}\n.CodeMirror {\n    border: 1px solid black;\n}\ninput.check {\n    color: white;\n    background-color: #4daf52;\n    border: 1px solid gray;\n    margin-bottom: 12px;\n    margin-right:auto;\n    margin-left:auto;\n    display: block;\n}\ninput.check:hover {\n    color: #4daf52;\n    background-color: white;\n    border: 1px solid gray;\n}\ninput[type=text] {\nmax-width: 300px;\n}\n\t.email-editor-dropzone-container {\n\t\tmargin:0;\n\t\twidth:90%;\n\t\t-webkit-box-sizing: border-box; \n\t\t-moz-box-sizing: border-box;    \n\t\tbox-sizing: border-box; \n\t\tmin-height:600px;\n\t\tborder: 2px solid gray;\n\t\tdisplay:inline-block;\nposition: relative;\n\t}\n\n\n\n\nnav.menu-intool{margin-top:2px;font-weight:700}nav.menu-intool ul{background:#4daf52;list-style:none;margin:0;padding:0;display:flex;flex-wrap:wrap;justify-content:center;text-align:center}nav.menu-intool ul li{position:relative;display:inline-block;margin-right:-4px;text-align:center;margin: 0em 0;}nav.menu-intool ul li a{display:block;background:#4daf52;padding:15px 20px;color:#fff;font-size:.85em;text-decoration:none;transition:.2s linear}nav.menu-intool ul li a.sub-menu-important{color:black;}nav.menu-intool ul li a:hover{background:white;cursor:pointer;color:#4daf52}nav.menu-intool ul li ul{position:absolute;top:100%;left:0;width:240px;border-bottom:4px solid #4daf52;flex-direction:column;display:flex}nav.menu-intool ul li ul li{display:block;width:100%;margin:0;text-align:left;z-index:1001}nav.menu-intool ul li ul li a{display:inline-block;padding:10px 15px}nav.menu-intool ul li a:hover{color:4daf52}nav.menu-intool ul li ul li a:hover{background:white;border-left:4px solid #4daf52;color:4daf52}.arrow{border:solid #fff;border-width:0 3px 3px 0;display:inline-block;padding:3px}.up{transform:rotate(-135deg);-webkit-transform:rotate(-135deg)}.down{transform:rotate(45deg);-webkit-transform:rotate(45deg)}.arrow-right{display:inline-block;overflow:hidden;position:relative;height:16px;width:8px;margin-left:15px}.arrow-right:before{border:8px solid transparent;content:' ';display:block;position:absolute;z-index:2;border-left-color:#4daf52;left:0;top:2px}\nnav.menu-intool ul li a ui a {display: inline-block;font-size:0.7em;}\n<\/style>\n\n\n\n<nav class=\"menu-intool animated flipInX\">\n\t<ul>\n\t\t<li class=\"sub-menu\">\n\t\t\t<a href=\"#\" >\n\t\t\t\tList of schemas\n\t\t\t\t<i class=\"arrow down\"><\/i>\n\t\t\t<\/a>\n\t\t\t<ul style=\"display: none;\" id=\"list-templates\">\n\t\t\t\t<!--li>\n\t\t\t\t\t<a id=\"menu-sqlite-openDB\">\n\t\t\t\t\t\tOpen DB file\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/li>\n\t\t\t\t<li>\n\t\t\t\t\t<a id=\"menu-sqlite-saveDB\" >\n\t\t\t\t\t\tSave DB file\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/li-->\n\t\t\t<\/ul>\n\t\t<\/li>\t\n\t\t<li class=\"sub-menu\">\n\t\t\t<a id=\"template-clone\">\n\t\t\t\tClone current schema\n\t\t\t<\/a>\n\t\t<\/li>\n\t\t<li class=\"sub-menu\">\n\t\t\t<a>\n\t\t\t\tImport \/ Export schema\n\t\t\t\t<i class=\"arrow down\"><\/i>\n\t\t\t<\/a>\n\t\t\t<ul style=\"display: none;\">\n\t\t\t\t<li>\n\t\t\t\t\t<a id=\"template-export\">\n\t\t\t\t\t\tExport current schema to file\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/li>\n\t\t\t\t<li>\n\t\t\t\t\t<a id=\"template-import\" >\n\t\t\t\t\t\tImport schema from file\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/li>\n\t\t\t<\/ul>\n\t\t<\/li>\n\t\t<li class=\"sub-menu\">\n\t\t\t<a href=\"#\" id=\"template-save\">\n\t\t\t\tSave current schema\n\t\t\t<\/a>\n\t\t<\/li>\n\t\t<li class=\"sub-menu\">\n\t\t\t<a href=\"#\" class=\"sub-menu-important\" id=\"generate\">\n\t\t\t\tGenerate \n\t\t\t<\/a>\n\t\t<\/li>\n\t<\/ul>\n<\/nav>\n<input type=\"file\" id=\"template-file\" style=\"display:none;\" \/>\n<div style=\"padding:20px;\">\n\n\tSchema name <input type=\"text\" id=\"template-name\" value=\"Your template\" \/>\n\n\n\t<div class=\"\">\n\t\t<div class=\"\">\n\t\t\t<div class=\"\" style=\"overflow-wrap: break-word; clear:both;\">\n\t\t\t\t<div class=\"editor-msg\">\n\t\t\t\t\t<div id=\"editor-valid\" style=\"display:none;\" class=\"editor-valid\"><\/div>\n\t\t\t\t\t<div id=\"editor-error\" style=\"display:none;\" class=\"editor-error\"><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div id=\"dropZone\" class=\"span\">\n\t\t\t\t\t<div class=\"columns\">\n\t\t\t\t\t\t<div class=\"csvField\">\n\t\t\t\t\t\t\t<div class=\"csvFieldData csvFieldButton csvFieldButtonMove\">\n\t\t\t\t\t\t\t\t<label class=\"csvFieldColumn\"><\/label>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<div class=\"csvFieldData csvFieldName\">\n\t\t\t\t\t\t\t\t<label class=\"csvFieldColumn\">Field title<\/label>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<div class=\"csvFieldData csvFieldType\">\n\t\t\t\t\t\t\t\t<label class=\"csvFieldColumn\">Data type<\/label>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<div class=\"csvFieldData\">\n\t\t\t\t\t\t\t\t<label class=\"csvFieldColumn\">Options<\/label>\n\t\t\t\t\t\t\t<\/div>\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t<div id=\"columns\" class=\"columns\"><\/div>\n\t\t\t\t\t<br\/>\n        \t                  <\/div>\n\t\t\t<\/div>\t\n\t\t<\/div>\n\t<\/div>\n<div style=\"clear:both;\"><\/div>\n\n\n\n\n\n\n<div>\n    <div class=\"container-center\">\n        <input type=\"button\" class=\"action-button-editor csv-add-field span\" id=\"addColumn\" value=\"Add field\"  \/>\n    <\/div>\n    <br\/>\n    <div class=\"container-center container-action\" style=\"clear:both\">\n        <label class=\"span\">Format:<\/label> \n        <select class=\"span select-action csvInputConfSmall\" id=\"format\"><option value=\"CSV\">CSV<\/option><option value=\"SQL\">SQL<\/option><option value=\"JSON\">JSON<\/option><option value=\"JSONL\">JSONL<\/option><option value=\"XML\">XML<\/option><\/select>\n        <label class=\"span\"># Rows: <\/label> \n        <input type=\"number\" class=\"span select-action csvInputConfSmall\" id=\"numberOfRow\" value=\"1000\" \/>\n        <label class=\"span\"># Files: <\/label> \n        <input type=\"number\" class=\"span select-action csvInputConfSmall\" id=\"numberOfFile\" value=\"1\" \/>\n        <label class=\"span\">Filename<\/label>\n        <input type=\"text\" class=\"span select-action\" id=\"filename\" value=\"myFile{{index()}}.csv\" \/>\n        <br\/><br\/>\n        <div id=\"XML-config\">\n            <label class=\"span\">Root tag<\/label> \n            <input type=\"text\" class=\"span select-action\" id=\"root\" value=\"root\" \/>\n            <label class=\"span\">Record tag<\/label> \n            <input type=\"text\" class=\"span select-action\" id=\"record\" value=\"record\" \/>\t\t\t\t\t\n        <\/div>\n        <div id=\"JSON-config\"><\/div>\n        <div id=\"SQL-config\">\n            <label class=\"span\">Table name<\/label> \n            <input type=\"text\" class=\"span select-action\" id=\"tablename\" value=\"MyTable\" \/>\n            <label class=\"span\">Include create table statement<\/label> \n            <select class=\"span select-action csvInputConfSmall\" id=\"createTable\"><option value='YES'>Yes<\/option><option value=\"NO\">No<\/option><\/select>\n        <\/div>\n        <div id=\"CSV-config\">\n            <label class=\"span\">Delimiter<\/label> \n            <select class=\"span select-action csvInputConfSmall\" id=\"separatorRow\"><option value=\",\">Comma<\/option><option value=\"\t\">Tab<\/option><option value=\";\">Semicolon<\/option><option value=\"|\" >Pipe<\/option><\/select>\n            <label class=\"span\">Quote<\/label> \n            <select class=\"span select-action csvInputConfSmall\" id=\"quoteRow\"><option value='\"'>Double quote<\/option><option value=\"'\">Quote<\/option><\/select>\n            <label class=\"span\">Escape<\/label> \n            <select class=\"span select-action csvInputConfSmall\" id=\"escapeRow\"><option value='\"'>Double quote<\/option><option value=\"'\">Quote<\/option><option value=\"\\\">Backslash<\/option><\/select>\n            <label class=\"span\">End line<\/label> \n            <select class=\"span select-action csvInputConfSmall\" id=\"endLine\"><option value=\"\\n\">\\n<\/option><option value=\"\\r\\n\">\\r\\n<\/option><\/select>\n            <br\/><br\/>\n            <label class=\"span\">Include header<\/label> \n            <select class=\"span select-action csvInputConfSmall\" id=\"headerCSV\"><option value='yes'>Yes<\/option><option value=\"no\">No<\/option><\/select>\n        <\/div>\n    <\/div>\t\t\t\t\t\n<\/div>\n<br\/>\n<br\/>\n<div id=\"container-result-first\" class=\"column-double span section-container\">\n    <div class=\"container-center\"><span class=\"section-title\">First file generated<\/span><\/div>\n    <input type=\"button\" class=\"action-button-editor\" value=\"Download\"  id=\"download\" \/>\n    <input type=\"button\" class=\"action-button-editor\" value=\"Copy to clipboard\"  id=\"copy\"\/>\t\n    <br\/>\n    <br\/><div class=\"row-container editor-container\" style=\"clear:both;max-width:1100px; width:100%;overflow-x: auto;\">\n        <textarea id=\"editor-container-result\" class=\"editor\"><\/textarea>\n    <\/div>\n<\/div>\n<div id=\"container-result\" class=\"column-double span section-container\" style=\"display:none;\">\n    <div class=\"container-center\"><span class=\"section-title\">All files generated<\/span><\/div>\n    <div id=\"container-result-files\" class=\"container-action container-result span\" style=\"display:none;\">\n    <\/div>\n<\/div>\n<br\/>\n\n<\/div>\n\n\n\n<h2 class=\"wp-block-heading deca\">How to generate test data<\/h2>\n\n\n\n<p class=\"deca\">This data generator allows you to create random test data by following these steps:<\/p>\n\n\n\n<ul>\n<li class=\"deca\"><strong>Define fields<\/strong>: Add the desired number of fields (by clicking the \"Add field\" button).<\/li>\n\n\n\n<li class=\"deca\"><strong>Name the fields<\/strong>: Assign a name to each field.<\/li>\n\n\n\n<li class=\"deca\"><strong>Choose the data type<\/strong>: Select the data type for each field (text, number, date, etc.).<\/li>\n\n\n\n<li class=\"deca\"><strong>Configure options<\/strong>: Fill in specific parameters if necessary.<\/li>\n\n\n\n<li class=\"deca\"><strong>Modify if needed<\/strong>: You can delete or rearrange fields.<\/li>\n\n\n\n<li class=\"deca\"><strong>Set the data format<\/strong>: Choose between CSV, SQL, XML, JSON, and JSONL!<\/li>\n\n\n\n<li class=\"deca\"><strong>Start generation<\/strong>: Click \"Generate.\"<\/li>\n\n\n\n<li class=\"deca\"><strong>Preview results<\/strong>: View a preview of a generated file.<\/li>\n\n\n\n<li class=\"deca\"><strong>Download or copy<\/strong>: Copy the data to the clipboard or download a ZIP file containing all CSV files.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading deca\">How to manage different test data models?<\/h2>\n\n\n\n<p class=\"deca\">You can save your models to generate data later based on these models without having to reconfigure all the columns. No account is required for this feature! \ud83d\ude0a<\/p>\n\n\n\n<ol>\n<li class=\"deca\"><strong>Enter a name<\/strong>: Fill in the \"Schema name\" field to name your model.<\/li>\n\n\n\n<li class=\"deca\"><strong>Save the model<\/strong>: Click the \"Save the current schema\" button to store your model.<\/li>\n\n\n\n<li class=\"deca\"><strong>Clone a model<\/strong>: Click \"Clone current schema\" to duplicate the current model and avoid starting from scratch.<\/li>\n\n\n\n<li class=\"deca\"><strong>Manage your schemas<\/strong>:\n<ul>\n<li class=\"deca\">In the <strong>\"List of schemas\"<\/strong> section, load a saved model by clicking on it.<\/li>\n\n\n\n<li class=\"deca\">Delete a model by clicking the cross icon.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading deca\">Data generator<\/h2>\n\n\n\n<p class=\"deca\"><span style=\"font-size: inherit;\">When developing web applications, there is often a need to create test data. Sometimes, it is necessary to populate a database or simulate an API to validate functionalities.<\/span> <\/p>\n\n\n\n<p class=\"deca\">I hope this basic yet effective data generator will be useful to you (Have you turned to the dark side of AI? ... Too long to generate large datasets \ud83d\ude09 &nbsp;). It allows you to quickly and easily create test data that closely resembles real data, which is essential for detecting bugs more easily by testing in conditions close to reality. Moreover, it is highly flexible and allows you to execute Python code (Brython) to generate data as needed.<\/p>\n\n\n\n<p class=\"deca\">This test data generator has no hard limit on the amount of data you generate. You can specify the number of rows you want, but you will be limited by the browser (and my code, which isn\u2019t very optimized...). In just a few weeks, you can generate tens of megabytes of data!<\/p>\n\n\n\n<h2 class=\"wp-block-heading deca\">Types of data generators<\/h2>\n\n\n\n<h3 class=\"wp-block-heading deca\">Test data generators<\/h3>\n\n\n\n<p class=\"deca\">Test data generators are primarily designed for software development and testing. They allow the creation of datasets, resembling real data to varying degrees, for testing applications.<\/p>\n\n\n\n<h3 class=\"wp-block-heading deca\">Synthetic data generators<\/h3>\n\n\n\n<p class=\"deca\">These generators produce \"artificial\" data that preserve the statistical properties\/distributions of real data. They maintain correlations between variables.<\/p>\n\n\n\n<h3 class=\"wp-block-heading deca\">Random data generators<\/h3>\n\n\n\n<p class=\"deca\">These tools create data with random values according to defined parameters. Useful for load testing and performance analysis, they quickly generate large volumes of data with flexibility in types and formats. They allow parameterization of value ranges and support various output formats such as CSV, JSON, or SQL. Random Data Generator and DTM Data Generator are typical solutions.<\/p>\n\n\n\n<h3 class=\"wp-block-heading deca\">Other types of generators<\/h3>\n\n\n\n<p class=\"deca\">There are other types of generators, such as those based on statistical models, others specific to a domain... There's something for every taste and need! Some may combine several types of generators.R\u00e9essayer<\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>List of schemas Clone current schema Import \/ Export schema Export current schema to file Import schema from file Save current schema Generate Schema name Field title Data type Options Format: CSVSQLJSONJSONLXML # Rows: # Files: Filename Root tag Record tag Table name Include create table statement YesNo Delimiter CommaTabSemicolonPipe Quote Double quoteQuote Escape Double [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/pythonium.net\/wp-json\/wp\/v2\/pages\/406"}],"collection":[{"href":"https:\/\/pythonium.net\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/pythonium.net\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/pythonium.net\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/pythonium.net\/wp-json\/wp\/v2\/comments?post=406"}],"version-history":[{"count":78,"href":"https:\/\/pythonium.net\/wp-json\/wp\/v2\/pages\/406\/revisions"}],"predecessor-version":[{"id":408,"href":"https:\/\/pythonium.net\/wp-json\/wp\/v2\/pages\/406\/revisions\/408"}],"wp:attachment":[{"href":"https:\/\/pythonium.net\/wp-json\/wp\/v2\/media?parent=406"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}