--- Log opened Sun Jul 05 00:00:56 2009
01:18 -!- penberg_home [n=penberg@cs149038.pp.htv.fi] has quit []
02:59 -!- tgrabiec [n=tomekg@abky38.neoplus.adsl.tpnet.pl] has quit [Remote closed the connection]
04:20 -!- penberg_home [n=penberg@cs149038.pp.htv.fi] has joined #jato
05:38 -!- penberg_home [n=penberg@cs149038.pp.htv.fi] has quit []
08:17 -!- Eduard_Munteanu [n=edi@79.115.194.110] has joined #jato
08:17 < Eduard_Munteanu> Hi.
09:11 -!- penberg_home [n=penberg@cs149038.pp.htv.fi] has joined #jato
09:14 < Eduard_Munteanu> penberg_home, good morning.
09:14 < penberg_home> Eduard_Munteanu: morning!
09:14 -!- mode/#jato [+o penberg_home] by ChanServ
09:16 <@penberg_home> Eduard_Munteanu: how are things? any problems?
09:17 < Eduard_Munteanu> penberg_home, I'm fixing some breakage. Will send some patches.
09:17 <@penberg_home> heh heh
09:17 <@penberg_home> master is moving pretty fast
09:18 <@penberg_home> we have fpu support now
09:18 <@penberg_home> and invokeinterface
09:20 < Eduard_Munteanu> Oh, so ahuillet did it.
09:21 <@penberg_home> yeah,
09:21 <@penberg_home> I think it's bit rough around the edges still
09:55 -!- tgrabiec [n=tomekg@abky38.neoplus.adsl.tpnet.pl] has joined #jato
10:11 -!- penberg [n=penberg@cs181039060.pp.htv.fi] has joined #jato
10:22 < penberg> It's kinda strange how little information is available on x87 code generation.
10:22 < penberg> Other than the obvious "it's hard" posts
10:27 < penberg> heh
10:27 < penberg> apparently there's a fxch instruction that's implemented interally as register renaming
10:28 < penberg> so you can "by-pass" the stack model
10:28 < penberg> http://www.smlnj.org//compiler-notes/x86-fp.ps
10:36 -!- tgrabiec [n=tomekg@abky38.neoplus.adsl.tpnet.pl] has quit [Read error: 110 (Connection timed out)]
10:37 -!- tgrabiec [n=tomekg@atq3.neoplus.adsl.tpnet.pl] has joined #jato
10:39 < Eduard_Munteanu> penberg, why not use that C* instruction to convert MMX->XMM? Seems supported on most machines.
10:40 < Eduard_Munteanu> i.e. you only need MMX and SSE1.
10:45 < penberg> Eduard_Munteanu: do we use mmx too somewhere?
10:46 < penberg> Eduard_Munteanu: but, don't ask me, ask ahuillet
10:46 < penberg> but MMX + SSE1 are obviously fine for us.
10:46 < penberg> that said, for x86-64, you should probably take full advantage of SSE2 and so on
10:47 < penberg> http://penberg.blogspot.com/2009/07/efficient-x87-code-generation.html
10:51 < Eduard_Munteanu> penberg, no, but MMX can be used so we don't access memory when doing a mov GPR, XMM.
10:52 <@penberg_home> Eduard_Munteanu: oh, nice.
10:52 < Eduard_Munteanu> That is, we can do mov GPR, MMX, then use that long C* instruction to convert MMX to XMM.
10:52 <@penberg_home> Eduard_Munteanu: I guess it's just an oversight from ahuillet's part
10:52 <@penberg_home> Eduard_Munteanu: having to tackle all the nasty issues it's easy to overlook something like that ;)
10:53 < Eduard_Munteanu> penberg_home, hm, no, I told him about that. He probably chose to do it the simple way first.
10:53 <@penberg_home> aah ok
10:53 <@penberg_home> in any case, mmx+sse1 is obviously fine if it generates better code
10:54 <@penberg_home> floating point performance is going to be very important for real world programs.
10:54 <@penberg_home> ok, benchmarks ;)
10:55 < Eduard_Munteanu> Well, I still could be wrong, I found that while browsing the AMD64 manuals. I don't know it for a fact :)
11:05 < tgrabiec> penberg_home: I'm getting sigsegv on symbol loading - on dlopen() when with RTLD_NOW, or on dlsym when with RTLD_LAZY  - http://pastebin.com/d2bc0b342
11:06 < tgrabiec> any ideas ?
11:06 <@penberg_home> master? or with some new patches?
11:06 < tgrabiec> oh, and i _don't_ get the sigsegv when jato is run without any -Xtrace:
11:07 < tgrabiec> penberg_home, master + initial jni, but it shouldn't be much relevant
11:07 <@penberg_home> curious
11:07 <@penberg_home> SIGSEGV at EIP b7ee2000 while accessing memory address b7ee2000.
11:07 <@penberg_home> that's in JIT code, isn't it?
11:07 < tgrabiec> i was thinkin that maybe it's related to aour new hanlding of jit buffers ?
11:08 <@penberg_home>  [<ffffe440>] <unknown>
11:08 <@penberg_home> that's strange.
11:08 < Eduard_Munteanu> It probably is in JITed code.
11:09 <@penberg_home> tgrabiec: to answer your question: no, I can't think of anything obvious
11:09 <@penberg_home> memory corruption caused by -Xtrace:jit?
11:09 < tgrabiec> penberg_home: is it possible that when .so is loaded it overwrites the memory allocated for JIT ?
11:10 <@penberg_home> not really.
11:10 <@penberg_home> we mmap() the memory so we own the virtual memory region
11:10 < Eduard_Munteanu> tgrabiec, no, it can also be a code emission problem. It can cause havoc later.
11:10 <@penberg_home> linker script problem?
11:10 <@penberg_home> (yes, just waving hands here)
11:12 < vegard> fixup_static
11:13 < vegard> at :1296 I have vm_class_ensure_init()
11:14 < Eduard_Munteanu> vegard, hm, that's causing some problems for me as well, although I'm quite sure it's my fault.
11:14 < tgrabiec> vegard: this could be because JNI object is loaded from some init code
11:15 < vegard> Eduard_Munteanu: oh. I wrote it, so feel free to ask any questions you might have about it :)
11:17 < vegard> tgrabiec: can we your patch so far?
11:17 < vegard> *see
11:17 < tgrabiec> vegard: i can send it, but it's quite dirty yet
11:20 < tgrabiec> here it is http://pastebin.com/d5c2bf646
11:24 < tgrabiec> (I'm testing it on new Integer(0))
11:25 < vegard> (btw, completely unrelated to your problem, I suggest looking at asprintf() for VMRuntime.mapLibraryName)
11:27 < tgrabiec> oh, thanks
11:36 < vegard> and wow, you've done quite a lot already
11:36 < tgrabiec> yeah, it would be also nice if it worked ;)
11:37 < vegard> so are you actually calling into any of the code loaded?
11:37 < vegard> *that has been
11:38 < tgrabiec> vegard: when there is no -Xtrace:invoke, it works - jni code is loaded and executed, it calls vm_jni_find_class()
11:39 < vegard> ah, wow, nice.
11:39 < vegard> ok, so what is the difference with invoke.
11:39 < tgrabiec> it emits a stub
11:40 < vegard> does the stub in any way conflict with the jni trampoline..?
11:42 < tgrabiec> it shouldn't since invoke puts stubs only for methods which have code emitted (not native)
11:42 < vegard> #
11:42 < vegard> +       jit_text_reserve(buffer_offset(buf));
11:42 < tgrabiec> yeah, I'd like to ask about that too
11:42 < vegard> what does this do?
11:43 < vegard> hm, emit_trampoline() does it too, so it's probably ok.
11:44 < tgrabiec> but there's something i've missed - buf->buf = jit_text_ptr();
11:44 < vegard> you shouldn't need it when you get struct buffer * passed to the function?
11:45 < vegard> ah
11:45 < vegard> I think you should probably allocate a new buffer...
11:45 < vegard> oh you are doing it.
11:46 < vegard> oh that jit_native_trampoline() looks a bit ugly :)
11:46 < tgrabiec> yeah, needs a cleanup
11:46 < vegard> I think you shouldn't be using alloc_buffer() to allocate your buffer.
11:47 < vegard> because that sets buffer ops to use realloc() etc when the buffer grows to big, iirc
11:50 < tgrabiec> vegard: thanks, it worked !
11:50 < vegard> :D
12:08 < vegard> wooow tests are really fast now :D
12:08 < vegard> reression tests, that is
12:09 < tgrabiec> vegard: what did you do ?
12:09 < vegard> open the zip files only once and keep them open
12:10 < tgrabiec> that's very cool :)
12:10 < vegard> real    0m0.072s
12:10 < vegard> for ArrayTest
12:11 <@penberg_home> faster than hotspot?
12:12 < vegard> yep. that one uses 0m0.094 but it doesn't even run the program because it can't bind the exit() native method
12:12 <@penberg_home> well
12:12 <@penberg_home> it probably _interprets_ the whole things too
12:13 <@penberg_home> with hotspot, jit kicks in only after N iterations
12:13 <@penberg_home> or N calls
12:13 <@penberg_home> where N is somewhere around 1000 maybe
12:13 < vegard> whoa :)
12:13 < vegard> are compilation costs that high?
12:13 <@penberg_home> with hotspot?
12:14 <@penberg_home> possibly
12:14 < vegard> yes, that it benefits from doing that
12:14 <@penberg_home> it does so many heavy optimizations
12:14 <@penberg_home> there's _two_ compilers in hotspot
12:14 <@penberg_home> and the "server" one is dog slow
12:14 <@penberg_home> but anyway
12:14 <@penberg_home> Jato is JIT only for a reason.
12:14 < vegard> time bash run-suite.sh is real 2.369s user 1.124s sys 1.208s
12:14 <@penberg_home> i want execution times to be predictable.
12:15 <@penberg_home> vegard: cool
12:15 < vegard> it hangs for a really long time in there though
12:15 <@penberg_home> I've seen that too
12:15 < vegard> ah no it doesn't. nvm.
12:15 <@penberg_home> aah ok
12:15 < vegard> it just appears that way because we don't get any warnings from a LOT of the tests
12:15 <@penberg_home> the slow version seems to do that
12:15 <@penberg_home> aah
12:16 <@penberg_home> makes sense
12:16 < vegard> ok. want the patch? :P
12:16 <@penberg_home> guess?
12:16 <@penberg_home> ;)
12:16 <@penberg_home> SHIP IT IN VEGARD THE GREAT!
12:17 < vegard> (hehe, better not mention that it was my bad code in the first place... oops!)
12:23 < penberg> nope
12:23 < penberg> and no need to mention the performance gains in the changelog!
12:23 < penberg> grr
12:23 < penberg> HAHA
12:23 < vegard> what, I can't take away _all_ your work ;)
12:23 < penberg> LOL
12:23 < penberg> penberg@penberg-laptop:~/src/linux-2.6$ git am -i -s ~/patch-queue/01
12:23 < penberg> ...wonder why it didn't apply?-)
12:24 < vegard> ...it didn't?
12:24 < penberg> holy crap! it's _fast_
12:24 < penberg> vegard: read that line carefully ;)
12:24 < penberg> apparently I'm getting old and stupid
12:24 < vegard> ah. wrong directory :)
12:25 < penberg> yes ;)
12:25 < penberg> it's SUPER fast now
12:26 < penberg> vegard: applied, thnx!
12:26 < vegard> how about adding echo $JAVA_CLASS to run-suite.sh ?
12:26 < penberg> vegard: sure and time too
12:27 < penberg> vegard: btw
12:27 < penberg> we can make "make regression" even faster
12:27 < penberg> if we javac all classes a the same time
12:27 < penberg> the biggest cost is the overhead of starting up that damned hotspot
12:27 < vegard> heh. not sure you'll believe me, but I just typed linux-kernel@vger.kernel.org in the Cc field... O.o
12:28 < penberg> I'm always afraid I do that
12:28 < penberg> it's hard-wired in the brain apparently
12:28 < Eduard_Munteanu> penberg, or use -j3 or whatever for make.
12:28 < vegard> yes.
12:28 < penberg> Eduard_Munteanu: on UP?
12:28 < penberg> make[1]: warning: jobserver unavailable: using -j1.  Add `+' to parent make rule.
12:29 < Eduard_Munteanu> penberg, oh, but it could work faster even on UP.
12:29 < Eduard_Munteanu> that is, if it's I/O-bound.
12:29 < vegard> that's a good point.
12:30 < penberg> Eduard_Munteanu: it's CPU bound
12:30 < penberg> the point here is that there's some constant overhead in _starting up_ hotspot
12:30 < penberg> which is why you usually want to compile multiple classes at the same time
12:30 < penberg> especially since it takes a while for hotspot to warm up
12:30 < penberg> so you reap the benefits from the JIT compilation and optimizations that way too
12:33 < vegard> btw, I think the "convention" of shell scripts is to use upper-case names for global (i.e. environment) variables and lower-case names for local variables
12:34 < penberg> vegard: yup
12:35 < penberg>     27.48%              jato  000000004000a9b6           0x0000004000a9b6
12:35 < penberg> perf + make regression
12:36 < vegard> tgrabiec: are you okay with my switching the license of your patch to cafebabe to be GPLv2 (no linking exception)?
12:36 < tgrabiec> vegard: feel free to modify
12:36 < penberg> hmmh
12:36 < penberg> vegard: so how is this going to work out?
12:36 < penberg> cafebabe is gplv2
12:37 < penberg> but when we merge to jato it's gplv2 + linking exception?
12:37 < vegard> yes.
12:37 < penberg> don't you need linking exception for cafebabe _anyway_ if you want to make it an usable _library_?
12:37 < vegard> not if the projects that use it are also GPL2 :)
12:37 < penberg>     47.03%              jato  00000000b80cb2c1           0x000000b80cb2c1
12:37 < vegard> penberg: what functions are those?
12:38 < penberg> I don't know!
12:38 < vegard> -Xperf ?
12:38 < penberg> yes!
12:38 < penberg> nothing coming out
12:39 < tgrabiec> vegard: oh, i have also done LineNumberTable support, I'll send the whole patchset now unless you see any problems with SourceFile patch
12:39 < vegard> btw. should we output perf entries for the invokeinterface resolution stubs?
12:39 < penberg> vegard: maybe
12:39 < vegard> tgrabiec: ah, okay, great :)
12:39 < penberg> for trampolines too
12:39 < penberg> but that's besides the point
12:39 < penberg> where the hell are we spending that time in?
12:40 < vegard> they'd probably not even show up in the perf though.
12:40 < vegard> penberg: do you have debuginfos for libzip ?
12:40 < penberg> hmm probably not
12:41 < penberg> nope
12:41 < penberg> no debug symbols available on ubuntu :(
12:41 < penberg> _but_
12:41 < penberg> perf should probably tell that it's in libzip
12:42 < vegard> does it appear for all the tests?
12:43 < penberg> it appears to be the same address for _all tests_
12:46 < vegard> well.
12:47 < vegard> if you do 'GDB=strace bash run-suite.sh jvm/ArrayExceptionsTest' you'll still see all those time(NULL) and stat64() calls.
12:47 < vegard> no, they are fewer, I just mean there are still lots of them.
12:49 < vegard> can you try to remove GLIBJ from the -cp and see if it makes a difference?
12:49 < vegard> and pick a test that doesn't need any of the classes from GLIBJ
12:51 < penberg> you can't remove glibj from classpath because of class preloading
12:52 < penberg> I sent an email to Ingo
12:52 < vegard> aah, true. and we need Object besides.
12:52 < penberg> you're cc'd. this seems to be problem in perf anyway
12:52 < penberg> the tool should be able to tell us what the hell is going on
12:52 < penberg> so tomek's patches
12:52 < penberg> should I just go ahead and merge them?
12:53 < vegard> I don't think so.
12:53 < vegard> give me an hour and I'll try to produce a cafebabe pull requests?
12:54 < penberg> fine
12:54 < penberg> the arrangement is not completely optimal IMHNSO
12:54 < penberg> one option is to just fix cafebabe licensing
12:54 < penberg> so you can backport the patches from jato
12:54 < penberg> so the flow is jato -> cafebabe
12:55 < penberg> as long as the comming only changes cafebabe code
12:55 < vegard> it's not that easy either
12:55 < penberg> the sha1 hash should be the same
12:55 < vegard> because I can't _merge_ from jato
12:55 < vegard> but you can merge from cafebabe
12:55 < vegard> and merging stuff is a lot better than applying a similar patch in two places
12:55 < penberg> well, I hope you see the obvious problems with the current approach
12:55 < vegard> because now git actually _knows_ that it's the same patch
12:55 < penberg> lets see if it's a one time thing or not
12:56 < penberg> git knows that even if it's applied twice
12:56 < penberg> that's magic of git
12:56 < penberg> but anyway, you know I'm a complainer
12:56 < vegard> no it doesn't? the sha1s will be different
12:56 < penberg> no they won't
12:56 < penberg> the sha1's are the same.
12:57 < penberg> I've done this _in practice_ already
12:57 < penberg> with -tip and slab.git
12:57 < vegard> when you merge, yes, not if you apply the same patch on two different parent sha1s
12:57 < penberg> I don't know why you keep saying that
12:57 < penberg> I just said _I've done it_ in the _real world_
12:57 < penberg> I sent a patch to Ingo which he merged
12:58 < penberg> which I had in slab.git
12:58 < vegard> yes, you probably both committed it on top of the same tree
12:58 < penberg> and the pulls Linus did were fine
12:58 < penberg> vegard: no?
12:58 < penberg> what's "the same tree"?
12:59 < vegard> the same commit
12:59 < penberg> on top of the same commit?
12:59 < penberg> of course not!
12:59 < vegard> you should keep in mind that I actually have to move the files in the patch before applying, so it's actually not even the same patch.
12:59 < penberg> that's irrelevant
12:59 < penberg> it's the _contents_ that matter AFAIK
12:59 < vegard> and the license change? :p
12:59 < penberg> yes, but I said (a) fix cafebabe licensing and (b) ...
13:00 < vegard> BTW, about your perf problem
13:00 < penberg> anyway, I just don't want to make cafebabe a neglected area of jato
13:00 < vegard> I guess perf doesn't even read the files in /usr/lib/debug ?
13:01 < penberg> because you have to jump through hoops to get patches merged
13:01 < penberg> vegard: thinks so?
13:01 < penberg> let me check
13:01 < penberg> but keep in mind
13:01 < penberg> that perf doesn't even seem to _think_ it's in /lib/ld.so
13:01 < penberg> the "Shared Object" part
13:01 < penberg> so that looks completely wrong too
13:01 < vegard> hm, yes.
13:01 < vegard> ok, true.
13:02 < penberg> util/symbol.c:			snprintf(name, size, "/usr/lib/debug%s", self->name);
13:02 < vegard> ahh, ok.
13:02 < penberg> so I do think it does the lookup
13:02 < penberg> but anyway, 40% is _huge_
13:03 < penberg> vegard: did you also notice that CPU utilization is only 50% or so for the regression tests
13:03 < penberg> that's bit strange isn't it?
13:05 < vegard> I have 25% usr 25% sys 50% idle (the 50% idle is the second core, probably)
13:05 < penberg> you do?
13:06 < vegard> that's what top says.
13:07 < vegard> oh
13:07 < vegard> wait.
13:07 < penberg> vegard: http://pastebin.com/m6cf2be2b
13:07 < penberg> UP
13:08 < penberg> aah I like Ingo so much!
13:08 < penberg> very responsive
13:09 < vegard> I'm trying to figure out whether you are being sarcastic or not.
13:09 < penberg> vegard: +1000 minor page faults, does that sound bit to you?
13:09 < penberg> vegard: sarcastic, how?
13:09 < penberg> vegard: I like the fact that you get replies from Ingo very quickly
13:10 < vegard> well, they're just minor, isn't that to be expected?
13:10 < vegard> I guess a few of them are our jit code emission?
13:11 < penberg> 1000 * 4 KB
13:11 < penberg> 3 megs?
13:11 < penberg> 3.9 megs
13:11 < vegard> minor = just vma sync (page table updates), isn't it?
13:12 < vegard> oh, no. I guess minor is copy-on-write ?
13:14 < penberg> vegard: http://en.wikipedia.org/wiki/Page_fault#Minor_page_fault
13:14 < penberg> ;)
13:14 < penberg> so it's cow after fork and the mmap() allocations
13:14 < penberg> but 4 megs? that's huge
13:24 < vegard> tgrabiec: there appears to be a problem with the SourceFile attribute patch
13:26 < vegard> but actually it's my fault... hm, we need to solve this problem cleanly.
13:26 < vegard> the point is that the utf8 constant-pool entries cannot just be cast into C strings
13:26 < vegard> but I noticed that we do it all over the place.
13:31 < vegard> hm. actually output to the console looks fine.
13:33 < vegard> aha. so the java utf8 is not an extension, just more restricted, than normal utf8
13:46 < vegard> ok, I know I said an hour, but I need a little bit more.
13:54 <@penberg_home> I'm not in a hurry
13:54 <@penberg_home> I'm compiling a new kernel
13:54 < vegard> ;-)
13:59 < vegard> so the JVM spec defines 9 attributes, we now parse 4 of them.
14:00 -!- penberg [n=penberg@cs181039060.pp.htv.fi] has quit ["leaving"]
14:19 < vegard> penberg_home: um, I think you forgot to add glibj.zip to the classpath?
14:19 <@penberg_home> ?
14:19 < vegard> ah, scripts/java does it?
14:20 <@penberg_home> yes
14:20 < vegard> ok.
14:20 -!- tgrabiec [n=tomekg@atq3.neoplus.adsl.tpnet.pl] has quit [Read error: 110 (Connection timed out)]
14:20 -!- tgrabiec [n=tomekg@aena189.neoplus.adsl.tpnet.pl] has joined #jato
14:33 -!- penberg [n=penberg@cs181039060.pp.htv.fi] has joined #jato
14:33 < penberg> vegard: http://pastebin.com/m7842d630
14:34 < penberg> vegard: looks like libzip
14:35 < vegard> but it's just 4.45% though?
14:36 < penberg> vegard: no elsewhere as well
14:36 < vegard> ah, not just the top
14:36 <@penberg_home>      4.27%              jato  /lib/tls/i686/cmov/libc-2.8.90.so  [.] __GI_strcmp
14:36 <@penberg_home> this is just crazy
14:37 <@penberg_home> where is it coming from?
14:37 <@penberg_home> or this
14:37 <@penberg_home>      4.08%              jato  /usr/lib/libzip.so.1.0.0   [.] _zip_get_name
14:37 <@penberg_home>      2.78%              jato  [kernel]                   [k] get_page_from_freelist
14:37 <@penberg_home> pretty interesting, eh?
14:37 < vegard> yes :)
14:37 <@penberg_home> vegard: the top three functions look fishy to mee
14:37 < vegard> well I suspect we could fix things with a libzip patch
14:38 < penberg> tgrabiec: mind resending your patches against current master?
14:38 < penberg> I pulled vegard's tree now
14:38 < tgrabiec> penberg: ok
14:38 < penberg> vegard: yeah, I suppose we need to import libzip then?
14:38 < penberg> (which we certainly can do)
14:39 < vegard> tgrabiec: tell me if you have any problems, ok?
14:39 < penberg> until the new version makes it to distros
14:39 < vegard> penberg: why? it's not like this 0.01s makes a difference until then, or? :-p
14:39 < vegard> so. do you want to download libzip and try my patch, or should I build a kernel that has "perf" support?
14:40 < penberg> vegard: use tip if you do the latter
14:40 < tgrabiec> penberg, vegard: I can see that cafebabe patches are already merged, right ?
14:40 < penberg> tgrabiec: yes
14:41 < penberg> vegard: I'll see if I can install libzip locally
14:45 < penberg> vegard: ok, everything set up
14:58 < vegard> can you get some more debug info now?
14:58 < vegard> you should compile it with -g I guess
15:02 < penberg> ok
15:02 < penberg> vegard: http://pastebin.com/m276564d7
15:03 < penberg> most of that is coming from localtime() though
15:03 < penberg> i've seen it to be terrible from performance pov before
15:04 < vegard> yes
15:04 < vegard> libzip shouldn't call it for every file in the zip file
15:04 < penberg> ->last_mod? what is this for.
15:04 < vegard> uh, where?
15:04 < penberg> _zip_dirent_read
15:04 < penberg>     dostime = _zip_read2(&cur);
15:04 < penberg>     dosdate = _zip_read2(&cur);
15:04 < penberg>     zde->last_mod = _zip_d2u_time(dostime, dosdate);
15:05 < vegard> last modified?
15:05 < penberg> yes
15:05 < vegard> _we_ don't use it at all
15:06 < penberg> yup
15:06 < penberg> it's just stupid
15:06 < penberg> libzip should do that conversion _lazily_
15:06 < penberg> libcrap....
15:06 < penberg> ;)
15:06 < vegard> so let's see, what is zip_d2u_time actually doing?
15:07 < penberg> converting the internal zip timestamp to time_t
15:07 < vegard> and why does it need syscalls for it?
15:07 < penberg> which is completely unnecessary
15:07 < penberg> localtime() <-
15:07 < vegard> couldn't it call that just once?
15:07 < penberg> ?
15:08 < penberg> ouchhh
15:08 < penberg> read the function carefully
15:08 < penberg> localtime() is not used at all!
15:08 < vegard> yes...
15:08 < vegard> I was wondering about it
15:08 < vegard> it's overwriting the fields anyway
15:08 < penberg> what the hell is it doing?
15:08 < vegard> no idea.
15:08 < vegard> try to remove the time() and localtime() lines?
15:09 < vegard> heh
15:09 < vegard> it only uses localtime to allocate a struct tm... lol
15:09 < penberg> yeah, that's what I'm about to try
15:09 < vegard> and it doesn't even free it
15:09 < vegard> afaics
15:09 < vegard> oh
15:09 < vegard> returns a static pointer.. ok
15:09 < vegard> sorry, pointer to static buffer
15:10 < penberg> http://pastebin.com/m5c6869d7
15:10 < penberg> I'm sending the patch to libzip maintainer
15:11 < vegard> you should check that it doesn't change anything _else_ though
15:11 < vegard> 24% what's that in?
15:13 < penberg> no idea
15:13 < vegard> do you have libc debuginfo? :B
15:13 < vegard> oh yeah.
15:13 < vegard> there are other functions in there which do show up...
15:15 < penberg> i do
15:18 < penberg> vegard: strcmp could come from _zip_headercomp
15:19 < penberg> vegard: should we abandon libzip and do our own thing insteaD?
15:19 < vegard> I don't think so.
15:20 < penberg> vegard: ok, so we fix it and import it our tree?
15:20 < vegard> I don't know how hard it would be to do it with just libz
15:20 < penberg> that's what I was thinking too
15:20 < vegard> but I simply don't "feel" like doing it
15:21 < vegard> it seems pointless. the library should be fixed instead.
15:21 < penberg> ok, it's not as if that's a huge issue anyway at this point
15:21 < vegard> I think you should wait to import it until we hear back from the maintainer
15:22 < penberg> whaddya mean?
15:22 < penberg> tgrabiec: applied thanks!
15:22 < penberg> #
15:22 < penberg>     24.15%              jato  /lib/tls/i686/cmov/libc-2.8.90.so  0x00000000007770
15:22 < penberg> this bothers me most
15:22 < vegard> < penberg> vegard: ok, so we fix it and import it our tree?
15:23 < vegard> it sounded like you wanted to import libzip in jato and apply your patch?
15:23 < penberg> yes.
15:23 < vegard> I don't think you should do that yet.
15:23 < penberg> sure we can wait
15:23 < vegard> it's a really tiny issue at the moment, and if we're lucky there will be a new version of libzip soon with the fix included.
15:24 < vegard> penberg: btw, how come you have libc-2.8.90 and not libc-2.9 ?
15:25 < vegard> (can I have a copy of your libc-2.8.90.so ?)
15:26 <@penberg_home> vegard: it's not going to make it to distros any time soon ;)
15:26 <@penberg_home> vegard: how can I send you that?
15:26 < vegard> penberg_home: can you put it online?
15:27 < vegard> or e-mail if you want, it shouldn't be big
15:27 <@penberg_home> http://www.laughingpanda.org/~penberg/libc-2.8.90.so
15:28 < vegard> thanks.
15:39 -!- ahuillet [n=ahuillet@AToulouse-158-1-130-16.w90-38.abo.wanadoo.fr] has joined #jato
15:40 < vegard> more e-mails to ingo?
15:42 <@penberg_home> whaddya mean?
15:42 <@penberg_home> think it's broken?
15:43 < vegard> what is the 0x7770, a virtual address, physical address, before or after relocation, etc.?
15:43 <@penberg_home> hmm
15:43 <@penberg_home> virtual of course
15:44 < vegard> objdump -D libc-2.8.90.so --start-address=0x7770 | head -n 20
15:45 < vegard> afaict 0x7770 is a _symbol table_
15:46 < vegard> but I probably didn't do the right thing.
16:13 < vegard> penberg_home: http://folk.uio.no/vegardno/0001-Switch-from-include-to-include-for-internal.patch
16:23 < Eduard_Munteanu> I'm trying to run regression tests manually, like this: $ ./jato -Xtrace:jit -cp $PWD:../runtime/classpath:/opt/gnu-classpath-0.97/share/classpath/glibj.zip regression/jvm/ExitStatusIsZeroTest.java
16:23 < Eduard_Munteanu> But it won't work: error: regression/jvm/ExitStatusIsZeroTest.java: could not load
16:23 < Eduard_Munteanu> Any ideas? There are 2-3 other warnings.
16:23 < vegard> you can't run .java files :p
16:24 < vegard> make sure you have the .class files and drop the extension
16:24 <@penberg_home> Eduard_Munteanu: use ./scripts/java -cp regression jvm/ExitStatusIsZeroTest
16:24 < Eduard_Munteanu> vegard, oh, okay, thanks :)
16:24 < Eduard_Munteanu> penberg_home, thanks.
16:26 <@penberg_home> vegard: yes?
16:26 <@penberg_home> vegard: send it properly and I'll merge it
16:26 < vegard> oh, ok. I thought it might be too big for the mailnig list. I guess not
16:26 <@penberg_home> vegard: send a pull request
16:35 < tgrabiec> penberg_home: I have a question about coding style - Vegard often mixes code and variable declarations, is it allowed ?
16:35  * vegard hides
16:35 <@penberg_home> tgrabiec: I am not a huge fan of that.
16:35 <@penberg_home> but there's no laws against it
16:35 <@penberg_home> we're c99
16:36 <@penberg_home> so I guess we can take advantage of it's features ;)
16:36 < tgrabiec> I'm asking because some of Vegard's constructions look appealing to me
16:36 <@penberg_home> tgrabiec: yeah?
16:36 < tgrabiec> yeah, sometimes it doesn't look good to have zillion of eclarations at the beggining of function
16:37 < tgrabiec> well, I know function should be short, but
16:37 <@penberg_home> yup
16:37 <@penberg_home> well, don't worry
16:37 <@penberg_home> I will complain if it gets out of control.
16:38 <@penberg_home> but you know, I am not a huge fan of rigid coding style rules either
16:38 <@penberg_home> as long as everything looks mostly consistent
16:38 <@penberg_home> and there's no surprises
16:38 <@penberg_home> i'm fine with it
16:45 < tgrabiec> ok, so I have preliminary JNI support working, I'll clean it up and send.
16:45 < vegard> :D
16:46 < vegard> how far does PrintTest get?
16:46 < tgrabiec> it deadlocks on getSecurityManager() invokation
16:46 < tgrabiec> I have no idea why, so I'll send JNI before debugging
16:46 < vegard> ok :) cool.
16:47 < penberg> vegard: pulled, testing...
16:48 < penberg> pushing...
16:49 < penberg> thnx
16:50 < vegard> hm. System.getSecurityManager() returns SecurityManager.current
16:51 < vegard> well, SecurityManager() constructor has an interesting comment.
16:52 < vegard> I'll leave it to you ;)
16:52 < Eduard_Munteanu> Is there any libzip corruption issue?
16:52 < Eduard_Munteanu> *memory corruption
16:53 < Eduard_Munteanu> I'm getting strange readings from valgrind.
16:53 < penberg> Eduard_Munteanu: none that we're aware of, no.
16:53 < penberg> vegard: ?
16:53 < vegard> ah. valgrind you can't use it
16:53 < Eduard_Munteanu> vegard, oh? Why?
16:54 < Eduard_Munteanu> I'm using it with --smc-check=all, so self-modifying code should be detected
16:54 < vegard> it isn't able to provide our signal handlers with detailed information about [RE]IP
16:54 < Eduard_Munteanu> Oh, I see, thanks.
16:54 < penberg> tgrabiec, vegard: hmm, any way to fix that?
16:55 < vegard> there was supposed to be a flag so that it would work, but it didn't make any difference for me
16:55 < tgrabiec> heh :)
16:55 < vegard> and I found some mailing list thread where the author of valgrind basically said "we won't fix it because it's not in posix"
16:55 < penberg> vegard, tgrabiec: -Xprint:invoke is pretty damn impressive for PrintTest even without JNI ;)
16:56 < penberg> vegard: aww crap
16:56 < penberg> It's puzzling to me why so many people choose to ignore reality over some paper standards :(
16:56 < penberg> that said, are we doing something fishy here?
16:56 < vegard> we could try to fix it. valgrind _is_ an incredibly useful tool (*hint hint*)
16:57 < penberg> yes, we _must_ get valgrind working sooner or later.
16:57 < Eduard_Munteanu> GTG now, bye.
16:57 < penberg> even if it means -Xvalgrind ;)
16:57 < penberg> Eduard_Munteanu: later.
16:57 -!- Eduard_Munteanu [n=edi@79.115.194.110] has quit ["Lost terminal"]
16:58 < vegard> btw, there's a "crocus" add-on for valgrind that checks signal handlers
16:59 < vegard> http://valgrind.org/downloads/variants.html?njn
16:59 < penberg> wow nice.
17:00 < penberg> I wonder why it hasn't been merged to mainline
17:02 < vegard> ==19723== Thread 1 return signal frame corrupted.  Killing process.
17:03 < vegard> that's what I get on trying to run GetstaticPatchingTest now... strange!
17:03 -!- ahuillet_ [n=user@AToulouse-158-1-130-16.w90-38.abo.wanadoo.fr] has joined #jato
17:06 < vegard> hm. looks like it pushes two magic constants (PI and E) on the stack
17:06 < vegard> before calling the handler
17:08 < vegard> or.. I don't know if it pushes it on the stack. but it's part of the signal "frame"
17:08 < vegard> I assumed that's a stack frame.
17:14 < vegard> what I don't get is this:
17:14 < vegard> valgrind is able to run signal handlers just fine and return to the EIP that faulted
17:15 < vegard> so why can't it _report_ the right one?
17:15 < vegard> hm. I guess it doesn't return to the real EIP, but to its own jit-compiled version?
17:16 -!- tgrabiec [n=tomekg@aena189.neoplus.adsl.tpnet.pl] has quit ["Leaving"]
17:20 < vegard> hm. valgrind seems to be reporting the __builtin_return_address(0) correctly.
17:22 < vegard> ahh
17:22 < vegard> its the memory address we accessed that it can't report correctly
17:22 < vegard> (apparently).
17:24 < vegard> hm, seems to be reported accurately too...
17:35 < vegard> *puzzled*
17:37 < vegard> sigsegv. si_addr = 0x437d000, guard_page = 0x437d000
17:38 < vegard> ok, so it definitely is capable of returning an accurate address for the memory we touched.
17:41 < vegard> the EIP is completely off, though.
17:42 < vegard> EIP = 0xa7c3f922
17:42 < vegard> EIP = 0x8872922
17:42 < vegard> that's without and with valgrind, respectively
17:44 < vegard> ok, I think the problem here is that valgrind puts the signal ucontext _on the stack_
17:46 < vegard> well, it should be
17:46 < vegard> but it's in the wrong place compared to normal execution.
17:48 < vegard> hm. maybe we should be using getcontext()/setcontext()
17:52 <@penberg_home> vegard: yeah, I wondered about that.
18:10 < vegard> I think we should.
18:13 < vegard> valgrind doesn't seem to be liking setcontext() from a signal handler
18:13 < vegard> it gives a lot of "invalid read" warnings
18:13 < vegard> but it _works_
18:19 < vegard> I also wonder if we really need the bottom handlers.
18:19 < vegard> the reason why those functions are unsafe to use in a signal handler is because the signal handler can be invoked at any random point in the program
18:20 < vegard> and the bottom handler won't change that
18:23 < vegard> hm. yes.
18:23 < vegard> just removing all the bottom handler stuff makes everything work.
18:23 < vegard> (for getstatic/putstatic patching, that is.)
18:24 < vegard> we should probably use setcontext() for exception handling.
18:30 < ahuillet_> http://penberg.blogspot.com/2009/07/efficient-x87-code-generation.html
18:30 < ahuillet_> oh.
18:30 < ahuillet_> are we going to do x87?
18:34 <@penberg_home> ahuillet_: dunno
18:34 <@penberg_home> at some point? sure.
18:34 <@penberg_home> now? probably not
18:35 <@penberg_home> ahuillet_: eduard asked why we don't use mmx + xmm btw?
18:35 < ahuillet_> why would we use mmx?
18:35 < ahuillet_> for what reason?
18:35 <@penberg_home> apparently you don't need to do load from memory
18:35 <@penberg_home> check the irc logs
18:35 < ahuillet_> you do
18:36 <@penberg_home> I forgot the details
18:36 <@penberg_home> http://folk.uio.no/vegardno/jato-irc-logs/
18:36 < ahuillet_> movq is SSE2.
18:36 < ahuillet_> movd as well
18:36  * penberg_home is off for a walk
18:36 <@penberg_home> later
18:37 < vegard> now I feel a bit bad for making tomek do all that signal handler bottom halves stuff.
19:20 < vegard> the setcontext/getcontext are hard to use though :-/
19:27 -!- tgrabiec [n=tomekg@aena189.neoplus.adsl.tpnet.pl] has joined #jato
19:34 < vegard> hey :)
19:36 -!- penberg_home [n=penberg@cs149038.pp.htv.fi] has quit []
19:36 < tgrabiec> hey
19:36 < ahuillet_> seems I have one regression test failing
19:40 < tgrabiec> vegard: there is another reason we needed bottom halfs - we can't call JIT code from signal, because if that JIT code triggers for example NullPointerExcpetion by another SIGSEGV, then system will not execute our handler, it will abort
19:41 < vegard> oh really?
19:41 < tgrabiec> yes - you can not nest SIGSEGV handlers
19:41 < vegard> hm, yes
19:41 < vegard> but when returning from the SIGSEGV
19:41 < vegard> we could return directly to the jit code?
19:42 < tgrabiec> vegard: well, that's what we do (we return to bottom half because it need to craete exceptions)
19:42 < vegard> create exceptions?
19:42 < vegard> hm, we can't do that, in signal handler or bottom half, I think.
19:43 < tgrabiec> why not?
19:43 < vegard> it needs to call malloc?
19:44 < tgrabiec> yes, but when signal was triggered from JIT code, we are sure that it is not in the middle of another malloc
19:44 < vegard> this doesn't work with threads
19:44 < tgrabiec> vegard: why not ?
19:47 -!- penberg_home [n=penberg@cs149038.pp.htv.fi] has joined #jato
19:49 < vegard> ok, I guess it works.
19:51 < tgrabiec> vegard: are you saying that setcontext() works with valgrind ?
19:51 < vegard> no, it actually doesn't.
19:52 < vegard> valgrind has its own magic signal handler stuff. but!
19:52 < vegard> remember you said that it reported the wrong ip, 16 bytes off? I can't really reproduce that.
19:54 < tgrabiec> what offset do you get?
19:54 < vegard> the ucontext [REG_IP] matches where it should
19:55 < tgrabiec> so what seems to be the problem?
19:55 < vegard> at the moment valgrind thinks we've corrupted the stack ;)
19:55 < vegard>    if (frame->magicPI != 0x31415927 ||
19:55 < vegard>        frame->magicE  != 0x27182818) {
19:55 < vegard>       VG_(message)(Vg_UserMsg, "Thread %d return signal frame "
19:55 < vegard>                                "corrupted.  Killing process.",
19:56 < tgrabiec> hmm, perhaps that's because we did. is 'frame' signal handler's frame or is it the frame in ucontext ?
19:57 < tgrabiec> oh, nvm
19:57 < vegard> it wants to restore the thread state from the frame, so I guess it's the ucontext's frame
20:48 < tgrabiec> with my latest patches, System.out.print(0) stops at: java/lang/ExceptionInInitializerError: java/io/PrintStream
21:07 < tgrabiec> i think that S.o.println is now blocked by the fact that Arrays (and Primitive type classes) do not "implement" Cloneable and Serializable interfaces.
21:07 < tgrabiec> vegard: that's your field right ?
21:08 < tgrabiec> something wants to do .clone() on array class
21:13 < penberg> tgrabiec: I've applied your patches!
21:16 < tgrabiec> there is a huge amount of work to do around JNI
21:17 < tgrabiec> I think I'll try to add exceptions support for it, and the rest will be added incrementally as needed
21:18 -!- penberg [n=penberg@cs181039060.pp.htv.fi] has quit ["leaving"]
21:54 < vegard> I can implement cloning... you didn't do it already, right?
21:54 < tgrabiec> no I didn't ;)
21:55 < tgrabiec> I think that since you wrote interfaces, it's best when you'll write this
22:00 < vegard> k, I'm on it
22:30 < vegard> tgrabiec: throw_from_native() will not return?
22:30 < tgrabiec> vegard: it will
22:31 < vegard> oh... I'm looking at native_vmobject_getclass()
22:31 < tgrabiec> oops..
22:31 < vegard> also, why are we checking !object->class there? isn't that unnecessary?
22:32 < vegard> what do we return instead? NULL?
22:32 < tgrabiec> well if object->class is NULL, we would crash
22:32 < vegard> but it should never happen.
22:32 < vegard> shouldn't that be assert() instead?
22:33 < tgrabiec> well, could be
22:34 < tgrabiec> 'll fix it
22:34 < vegard> it's just that the way it was, it looked like an object could somehow have a NULL class
22:36 < tgrabiec> sent
22:54 < vegard> hm. I guess clone() on array still returns just a shallow object.
22:55 < vegard> sorry
22:55 < vegard> shallow clone
22:59 < vegard> "For primitve type classes this holds a vm_type"
22:59 < vegard> vm/class.h
22:59 < vegard> ;)
23:15 -!- penberg_home [n=penberg@cs149038.pp.htv.fi] has quit []
--- Log closed Mon Jul 06 00:00:56 2009