Submitted by : simon at: 2007-05-02T17:35:36+00:00 (10 years ago)
Name :
Category : Severity : Status :
Optional subject :  
Optional comment :

(10:21:42) kosh: I trust zope to save things, however I setup the data so
that it is more efficient to save
(10:21:46) kosh: so I worry about what really matters
(10:21:59) peterb1: like relational data?
(10:22:13) kosh: no
(10:22:22) tiredbones left the room (quit: Read error: 60 (Operation timed out)).
(10:22:48) kosh: for instance don't store a whole bunch of basic attributes (strings,
lists, dicts etc) inside an object you will have to load a lot if you don't need all the
items inside it all the time
(10:23:14) xbeanx: good tip
(10:23:14) kosh: the zodb will load up all second class objects when the parent
is opened but all objects that inherit from persistent are only loaded when they are
accessed directly
(10:23:29) kosh: stuff like that REALLY cuts down on disk io to load objects
(10:23:44) kosh: another thing is only write a var back to itself if it has changed
(10:23:53) jbb666 left the room (quit: ).
(10:23:55) kosh: that speeds up saving and dramatically slows db growth
(10:24:22) xbeanx: Are most zope developers still developing for Zope 2?
(10:24:25) sm: good point.. kind of shame to break object encapsulation though, if
something belongs to an object and you store it elsewhere
(10:24:40) kosh: ie if you just keep hitting the save button you should not be doing io
in general
(10:24:43) sm: you have to keep them in sync, etc.
(10:24:50) kosh: sm: no reason to break encapsulation, just put it in a subobject
(10:25:00) sm: aha
(10:25:06) sm: interesting!
(10:25:10) kosh: sm: for instance you put metadata for a picture directly in your object
but the file data as a subobject
(10:25:26) kosh: sm: that way you can get the metadata for display purposes easily but only
load the file when it is actually requests
(10:25:44) sm: this is something to think about for zwiki page revisions
(10:25:48) peterb1: like extfile :)
(10:26:22) CrippsFX [n=Cripps@leonid2.housing.mun.ca] entered the room.
(10:26:23) xbeanx: kosh: interesting, I'm actually working with image objects right now..
(10:26:26) ***sm thinks.. guess the way it is now is also fine
(10:26:33) kosh: other simple thing like doing write lots of changes to a dict
(10:26:41) kosh: if you make a change to a simple object the entire object is saved again
(10:27:03) CrippsFX left the room (quit: Client Quit).
(10:27:07) kosh: so if you write several hundred changes to a dict one transaction at a time
you are looking at probably 100M or more of memory usage
(10:27:16) kosh: however if you used an OOBTree you are probably about 2M
(10:27:18) CrippsFX [n=Cripps@leonid2.housing.mun.ca] entered the room.
(10:27:34) kosh: it is because an OOBTree can write changes while a dict works like that
penny a day doubled thing
(10:28:09) sm: I wonder if I could halve zwiki memory usage by saving the prerendered
text as a subobject instead of an attribute
(10:28:57) kosh: probably help a fair bit
(10:29:03) sm: OOBTree.. where would you use that ? as a container?
(10:29:09) kosh: however it is probably even better to make them both subobjects
(10:29:20) kosh: that way to load one you never have to load the other
(10:29:34) kosh: anyplace you would use a dict you can use an OOBTree

summary: zwiki page objects could be split into more zodb objects, so that eg touching a page object to check it's attributes doesn't require loading up the prerendered data from zodb.