bugGNU Health - Bugs: bug #64542, health_genetics update take too...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #64542: health_genetics update take too long time

Submitter:  Feng Shu <tumashu>
Submitted:  Fri 11 Aug 2023 08:31:04 AM UTC
   
 
Category:  Database Severity:  3 - Normal
Item Group:  None Status:  Ready For Test
Privacy:  Public Assigned to:  meanmicio
Open/Closed:  Open Release:  4.3.0
Module:  health_genetics Component:  Hospital Management

Jump to the original submission

Wed 13 Sep 2023 08:40:12 AM UTC, comment #10: 

Wonderful!

Thank you! Much cleaner now :)

Bests
Luis

Luis Falcon <meanmicio>
Group administrator
Wed 13 Sep 2023 01:45:08 AM UTC, comment #9: 

updated

comment #8:

> I noticed that what it takes a lot is the update of the translation strings.
>
> I propose, by now, not translate the gene long name. Having the gene long name description in English is OK in the scientific community, and it will make the update process much faster, and don't overload the translation server at Weblate.
>
>
> All the entries with "model:gnuhealth.disease.gene,long_name:"
> should be removed, and make the field not translatable.
>
> For instance:
> msgctxt "model:gnuhealth.disease.gene,long_name:"
> msgid "5-methyltetrahydrofolate-homocysteine methyltransferase reductase"
> msgstr ""
>
> This should be removed from es.po, zh_CN.po ....
>
> Let me know your thoughts
>
> Bests
> Luis
>

Feng Shu <tumashu>
Mon 11 Sep 2023 04:20:34 PM UTC, comment #8: 

I noticed that what it takes a lot is the update of the translation strings.

I propose, by now, not translate the gene long name. Having the gene long name description in English is OK in the scientific community, and it will make the update process much faster, and don't overload the translation server at Weblate.


All the entries with "model:gnuhealth.disease.gene,long_name:"
should be removed, and make the field not translatable.

For instance:
msgctxt "model:gnuhealth.disease.gene,long_name:"
msgid "5-methyltetrahydrofolate-homocysteine methyltransferase reductase"
msgstr ""

This should be removed from es.po, zh_CN.po ....

Let me know your thoughts

Bests
Luis

Luis Falcon <meanmicio>
Group administrator
Tue 15 Aug 2023 05:05:51 PM UTC, comment #7: 

comment #6:

> I do not know why, maybe we should wait other's fallback.


Agree. In the meantime, please create a new, clean DB and run trytond-admin --update=health_genetics with the -vv argument.

Send the output of the time spent on each package datafile update, as I've done in previous messages.

Thanks!
Luis




Luis Falcon <meanmicio>
Group administrator
Tue 15 Aug 2023 12:39:57 AM UTC, comment #6: 


comment #5:

> That is strange... something is wrong with your box / DB...


I do not know why, maybe we should wait other's fallback.

Feng Shu <tumashu>
Mon 14 Aug 2023 10:56:30 AM UTC, comment #5: 

That is strange... something is wrong with your box / DB...

On my side, with the grouped attribute set on datafiles, the update of health_genetics package took 1 minute and 30 seconds (and is a basic laptop).

Have you tried a "clean" db? The current one might be fragmented.. Are you sure you're doing it with the grouped attribute?

Bests,
Luis


79129 35048148992 [2023-08-14 10:48:08,100] INFO trytond.modules health_genetics:register gnuhealth.gene
79129 35048148992 [2023-08-14 10:48:08,175] INFO trytond.modules health_genetics:register gnuhealth.protein.disease
79129 35048148992 [2023-08-14 10:48:08,229] INFO trytond.modules health_genetics:register gnuhealth.gene.variant
79129 35048148992 [2023-08-14 10:48:08,264] INFO trytond.modules health_genetics:register gnuhealth.gene.variant.phenotype
79129 35048148992 [2023-08-14 10:48:08,290] INFO trytond.modules health_genetics:register gnuhealth.patient.genetic.risk
79129 35048148992 [2023-08-14 10:48:08,338] INFO trytond.modules health_genetics:register gnuhealth.patient.family.diseases
79129 35048148992 [2023-08-14 10:48:08,362] INFO trytond.modules health_genetics:register gnuhealth.patient
79129 35048148992 [2023-08-14 10:48:20,123] INFO trytond.modules health_genetics:loading health_genetics_view.xml
79129 35048148992 [2023-08-14 10:48:44,404] INFO trytond.modules health_genetics:loading data/human_genes_grch38.xml
79129 35048148992 [2023-08-14 10:49:39,486] INFO trytond.modules health_genetics:loading data/human_genes_alt_loci.xml
79129 35048148992 [2023-08-14 10:49:39,620] INFO trytond.modules health_genetics:loading data/gnuhealth_commands.xml
79129 35048148992 [2023-08-14 10:49:39,658] INFO trytond.modules health_genetics:loading security/access_rights.xml
79129 35048148992 [2023-08-14 10:49:39,774] INFO trytond.modules health_genetics_uniprot

Luis Falcon <meanmicio>
Group administrator
Mon 14 Aug 2023 09:50:09 AM UTC, comment #4: 


comment #3:

> Fixed on
> https://hg.savannah.gnu.org/hgweb/health/rev/69ee52cd6836
>


I have test again, slowly still.

I try

   po-export.sh zh_CN

need about 15 min, after this, I run

  ./trytond-admin -d po-export-db -u health_genetics

need > 50 min

I do not know why.

Feng Shu <tumashu>
Fri 11 Aug 2023 11:38:56 AM UTC, comment #3: 

Fixed on
https://hg.savannah.gnu.org/hgweb/health/rev/69ee52cd6836

Just putting the delta for different datafiles load time on my laptop. Major change.

[2023-08-11 11:20:20,982] INFO trytond.modules health_genetics:loading data/human_genes_grch38.xml
[2023-08-11 11:22:19,337] INFO trytond.modules health_genetics:loading data/human_genes_alt_loci.xml
[2023-08-11 11:22:19,539] INFO trytond.modules health_genetics:loading data/gnuhealth_commands.xml
31230 38979963904 [2023-08-11 11:22:19,560] INFO trytond.modules health_genetics:loading security/access_rights.xml
31230 38979963904 [2023-08-11 11:22:19,663] INFO trytond.modules health_genetics_uniprot
31230 38979963904 [2023-08-11 11:22:19,663] INFO trytond.pool setup pool for "sandbox43"
31230 38979963904 [2023-08-11 11:22:19,664] INFO trytond.modules health_genetics_uniprot:loading data/protein_diseases.xml
31230 38979963904 [2023-08-11 11:22:29,241] INFO trytond.modules health_genetics_uniprot:loading data/gene_variants.xml
31230 38979963904 [2023-08-11 11:28:08,173] INFO trytond.modules health_genetics_uniprot:loading data/variant_phenotypes.xml

Luis Falcon <meanmicio>
Group administrator
Fri 11 Aug 2023 10:27:35 AM UTC, comment #2: 

We can set the grouped attribute (grouped="1") on the large datafiles, as we have it on icd10. That should speed up loading.

Bests
Luis


Luis Falcon <meanmicio>
Group administrator
Fri 11 Aug 2023 09:12:16 AM UTC, comment #1: 


original submission:

>    ./trytond-admin -d gnuhealth -u health_genetics
>
> and find it take too long time, any way improve it ?


IIRR there was a change in the Tryton server on how to deal with (large) static content that causes the loooong runtimes.
Not sure GNU Health can do anything on that w/o patching the server

Axel Braun <coogor>
Fri 11 Aug 2023 08:31:04 AM UTC, original submission:  

I try command:

   ./trytond-admin -d gnuhealth -u health_genetics

and find it take too long time, any way improve it ?

Feng Shu <tumashu>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by meanmicio (Posted a comment)
  • -email is unavailable- added by coogor (Posted a comment)
  • -email is unavailable- added by tumashu (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-08-11 meanmicio CategoryNone Database
        StatusNone Ready For Test
        Assigned toNone meanmicio
    2023-08-11 meanmicio ReleaseNone 4.3.0

    Back to the top

    Powered by Savane 3.13-02a9.
    Corresponding source code