Skip to content

Issue 37 we need a simple module that crawl the site posts every minute#55

Open
fahimi85 wants to merge 2 commits intomasterfrom
issue-37-We_need_a_simple_module_that_crawl_the_site_posts_every_minute
Open

Issue 37 we need a simple module that crawl the site posts every minute#55
fahimi85 wants to merge 2 commits intomasterfrom
issue-37-We_need_a_simple_module_that_crawl_the_site_posts_every_minute

Conversation

@fahimi85
Copy link
Copy Markdown
Contributor

In this issue can save all new posts every one minutes in post table.

@mostafa8026
Copy link
Copy Markdown
Member

کانفلیکت رو بگیر که مرج بشه

Copy link
Copy Markdown
Contributor

@we4netbot we4netbot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

سلام مستر فهیمی

من هنگام استرات پروژه این ارور برام میاد:

E:\diffal\node_modules\typeorm\error\TypeORMError.js:9
        var _this = _super.call(this, message) || this;
                           ^

QueryFailedError: invalid input syntax for type uuid: "gYsodNDt"
    at QueryFailedError.TypeORMError [as constructor] (E:\diffal\node_modules\typeorm\error\TypeORMError.js:9:28)
    at new QueryFailedError (E:\diffal\node_modules\typeorm\error\QueryFailedError.js:13:28)
    at PostgresQueryRunner.<anonymous> (E:\diffal\node_modules\typeorm\driver\postgres\PostgresQueryRunner.js:250:31)
    at step (E:\diffal\node_modules\tslib\tslib.js:143:27)
    at Object.throw (E:\diffal\node_modules\tslib\tslib.js:124:57)
    at rejected (E:\diffal\node_modules\tslib\tslib.js:115:69)
    at processTicksAndRejections (node:internal/process/task_queues:96:5) {
  query: 'SELECT "PostEntity"."id" AS "PostEntity_id", "PostEntity"."title" AS "PostEntity_title", "PostEntity"."description" AS "PostEntity_description", "PostEntity"."notes" AS "PostEntity_notes" FROM "post" "PostEntity" WHERE "PostEntity"."id" IN ($1)',
  parameters: [ 'gYsodNDt' ],
  driverError: error: invalid input syntax for type uuid: "gYsodNDt"
      at Parser.parseErrorMessage (E:\diffal\node_modules\pg-protocol\dist\parser.js:287:98)
      at Parser.handlePacket (E:\diffal\node_modules\pg-protocol\dist\parser.js:126:29)
      at Parser.parse (E:\diffal\node_modules\pg-protocol\dist\parser.js:39:38)
      at Socket.<anonymous> (E:\diffal\node_modules\pg-protocol\dist\index.js:11:42)
      at Socket.emit (node:events:390:28)
      at addChunk (node:internal/streams/readable:315:12)
      at readableAddChunk (node:internal/streams/readable:289:9)
      at Socket.Readable.push (node:internal/streams/readable:228:10)
      at TCP.onStreamRead (node:internal/stream_base_commons:199:23) {
    length: 140,
    severity: 'ERROR',
    code: '22P02',
    detail: undefined,
    hint: undefined,
    position: undefined,
    internalPosition: undefined,
    internalQuery: undefined,
    where: "unnamed portal parameter $1 = '...'",
    schema: undefined,
    table: undefined,
    column: undefined,
    dataType: undefined,
    constraint: undefined,
    file: 'uuid.c',
    line: '137',
    routine: 'string_to_uuid'
  },
  length: 140,
  severity: 'ERROR',
  code: '22P02',
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: "unnamed portal parameter $1 = '...'",
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'uuid.c',
  line: '137',
  routine: 'string_to_uuid'
}

که بنظرم مربوط میشه به

crawel.service.ts

@Cron('60 * * * * *')
  async handleCron() {
    const categorylist = [
      'انتخاب دسته بندی',
      'buy-old-house',
      'buy-villa',
      'buy-apartment',
      'buy-residential',
      'auto',
      'classic-car',
      'rental-car',
      'heavy-car',
      'electronic-devices',
      'home-kitchen',
      'services',
      'personal-goods',
      'entertainment',
      'social-services',
      'tools-materials-equipment',
      'jobs',
    ];
    const category = categorylist[1];
    const response = await got(
      `https://api.divar.ir/v8/web-search/mashhad/${category}`,
    );

    const json = JSON.parse(response.body);
    const Adverlist: { title: string; descript: string }[] = [];
    json.widget_list.forEach(
      (item: {
        data: { token: string; title: string; description: string };
      }) => {
        const tokenid = item.data.token;
        const Adverobject = {
          title: item.data.title,
          descript: item.data.description,
        };
        this.postService.update(tokenid, Adverobject);
        Adverlist.push(Adverobject);
      },
    );
    // console.log('Count Advertice :', Adverlist.length);
    // console.log(Adverlist);
  }
}

این خط

this.postService.update(tokenid, Adverobject);
        Adverlist.push(Adverobject);

@fahimi85

@mostafa8026
Copy link
Copy Markdown
Member

کانفلیکت داره

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants