Re: 5291 RTF import slow - the solution :-)

From: Dom Lachowicz (domlachowicz@yahoo.com)
Date: Wed Sep 24 2003 - 08:32:10 EDT

  • Next message: Hubert Figuiere: "commit: (HEAD) fix bug 5795"

    --- msevior@physics.unimelb.edu.au wrote:
    > >
    > > The present system has two real weaknesses; one is
    > the need to do
    > > string comparisons when comparing properties, and
    > the other is the fact
    > > that we have to clone the string values when
    > making copies,
    > > passing properties through functions, etc. If the
    > properties were stored
    > > internally in a numerical format both of these
    > would go away. I think
    > > Hub's proposal should be considered.
    > >
    > > Tomas
    >
    > The problem is in pt_VarSet.cpp::addIfUnique(...)
    >
    > If instead of requiring each index to be unique we
    > just add every property
    > set we get a dramatic increase in loading speed at
    > the cost of extra
    > memory usage.

    My question here is: what are the "before" and 'after"
    memory usages and times? If we're doing all of this
    comparison for essentially nothing, then maybe this
    should go away. But then again, if the memory bloat is
    like an extra few MB, then I'd be reluctant to change
    it.

    And we do *not* want to gzip the strings.

    1) It'll be slow
    2) The strings won't be readable in the debugger
    3) It'll increase memory usage because the strings are
    too short to have any good entropy - as well as all
    the boundary points where we get things out or put
    things in - those will need uncompressing + strdup'ing

    Dom

    __________________________________
    Do you Yahoo!?
    Yahoo! SiteBuilder - Free, easy-to-use web site design software
    http://sitebuilder.yahoo.com



    This archive was generated by hypermail 2.1.4 : Wed Sep 24 2003 - 08:46:51 EDT