Commit Graph

81 Commits

Author SHA1 Message Date
Masamichi Takagi
25ef4e9261 Merge branch 'postk_master' into development
* Merge 53e436ae7db1ed457692dbe16ccb15511aa6bc64
* Only arm64 stuff are left

Change-Id: I6b79de1f659fa61e75f44811b639d41f9a37d6cc
2019-02-01 15:14:58 +09:00
Dominique Martinet
960a6f5f90 prepare process: add magic header in program_load_desc
Check we mapped the correct region with a magic header in the struct

Original commit: d246b93a3bced92d0ac2a4a337118091b010658a

Fujitsu: POSTK_DEBUG_TEMP_FIX_76
Change-Id: If848be64af5d76844ba65b48493021637c8114f4
2019-02-01 13:16:25 +09:00
Ken Sato
dfd23c3ebe prctl: Add support for PR_SET_THP_DISABLE and PR_GET_THP_DISABLE
Change-Id: I04c5568a9eb78bcac632b734f34bba49cf602c4d
Refs: #1181
2019-01-22 05:40:56 +00:00
Shiratori, Takehiro
fd8bed670e ihk_os_setperfevent: Return number of registered events
In addition to that, mcctrl_perf_set is modified so that it updates
usrdata->perf_event_num with number of registered events.

Change-Id: I3f343176f55b06d3baab0b0fe34e240f39706cf6
Fujitsu: POSTK_DEBUG_TEMP_FIX_80
2018-10-24 06:16:41 +00:00
Dominique Martinet
b1681f4a3a mcexec/execve: fix shebangs handling
There were mainly two problems with shebangs:
 - Suffix arguments handling e.g. '#!/bin/sh -x'
 - Recursive handling e.g. script1 fetchs '#!/path/to/script2'
and script2 itself has a shebang
 - (did I say two?) running shebang would replace argv[optind] instead
of appending e.g. script with '#!/bin/sh' and running './script -c'
would run '/bin/sh -c' instead of '/bin/sh ./script -c'

There also are two places where this needs parsing:
 - starting a fresh program from mcexec
 - starting a new program from execve in mcexec

The first was easy to fix as we already had argv around, but the later
required a new way to transfer the 'new argv elements from the script'
to mckernel to append before its argv -- it used to be 'desc->shell_path'
but that was no longer used at some point and just one keyword is not
enough to handle this properly.

This commit does:
 - Refactors the lookup_path + load_elf_desc that was only done at most
twice in its own function that loops indefinitely and use that in both
situations described above
 - Transmits the argv addition in the transfer to mckernel after the
desc; mckernel allocates 4 pages (hardcoded) for the descs and we will
hopefully have room for the script arguments on top of that... (there is
no guard!!!)
 - Change flatten_strings to allow prepending a flattened string instead
of a single string.
Note that the flatten_string change also brought in a difference in the
format, to have the full length embedded within the string, the latest
slot that used to be zeroes now contains the position of the end of the
buffer (where the last+1 string would be if there had been one)
This required a trivial change in mckernel prepare args function that
used this property for no real reason.

Hopefully things work™, this probably warrants adding a couple of new
ostests...
 - create a couple of scripts with recursive invocation/arguments and
check their own argv.
 - execute "mcexec script args" and "mcexec sh -c 'script args'"

Change-Id: I2cf9cde5c07c9293f730de89c9731bd93dbfa789
Refs: #1115
2018-10-04 14:31:02 +09:00
Tomoki Shirasawa
7e342751a2 do_syscall: Delegate system calls to the mcexec with the same pid
This includes the following fix:
send_syscall, do_syscall: remove argument pid

Fujitsu: POSTK_TEMP_FIX_26
Refs: #1165
Change-Id: I702362c07a28f507a5e43dd751949aefa24bc8c0
2018-09-13 16:59:47 +09:00
Masamichi Takagi
8900c2cec5 uti: mcexec_uti_attr: Fix CPU binding decision
Change-Id: I4047858895503ae912e5575bb232dbbb2f915722
2018-09-04 19:53:03 +09:00
Masamichi Takagi
781a69617b uti: Replace data types represented as arrays with C structures
Defining C structures for the following objects:
(1) Remote and local context
(2) Stack of system call arguments / return values

Change-Id: Iafbb6c795bd765e3c78c54a255d8a1e4d4536288
2018-09-04 19:53:03 +09:00
Masamichi Takagi
98ee584ab6 uti: Change field name of release_user_space_desc
Change-Id: I18ada86ec3835198c1a947d8ceb36075d6ff2e94
2018-09-04 19:53:02 +09:00
Masamichi Takagi
e42c414454 uti: Hook system calls by binary-patching glibc
(1) Add --enable-uti option. The binary-patch library is
    preloaded with this option.
(2) Binary-patching is done by syscall_intercept developed by Intel

This commit includes the following fixes:

(1) Fix do_exit() and terminate() handling
(2) Fix timing of killing mcexec threads when McKernel thread calls terminate()

Change-Id: Iad885e1e5540ed79f0808debd372463e3b8fecea
2018-09-04 19:53:02 +09:00
Masamichi Takagi
5cb8a1f10f uti: Workaround not to share CPU with OpenMP threads
* Assign uti thread to the last idle CPU so that it's not shared with
  an OpenMP thread

Change-Id: Ia42cae056ce81fde9b6dab6286b39a52f3c9e172
2018-09-04 19:53:01 +09:00
Masamichi Takagi
b6ab5911b7 uti: Identify uti thread by clone count
--uti-thread-count <count> is added to mcexec.

Change-Id: Id9ec464412a5bb71e4d9e87d05f79de22d35b067
2018-09-04 19:53:01 +09:00
Masamichi Takagi
c71291a429 mcctrl: Add mcexec_terminate_thread_unsafe()
Change-Id: I6ca54cdac2ab9449d40b22f7329f1a215e5aa33b
2018-09-04 19:52:13 +09:00
Masamichi Takagi
63703589e5 uti: Clear user space PTEs after first fork in create_tracer()
Change-Id: I60755f0cb5e84c3a5a5cd91515411a30f0995822
2018-09-04 19:52:12 +09:00
Masamichi Takagi
52afbbbc98 uti: Call into McKernel futex()
(1) Masquerade clv
(2) Fix timeout
(3) Let mcexec thread with the same tid as McKernel thread migrating
    to Linux handles the migration request
(4) Call create_tracer() before creating proxy related objects

Change-Id: I6b2689b70db49827f10aa7d5a4c581aa81319b55
2018-09-04 19:52:10 +09:00
Balazs Gerofi
a697f5e98d partitioned execution: pass process rank to LWK
Cherry-pick of d2d134d5e6a4b16a34d55d31b14614a2a91ecf47

Conflicts:
	kernel/include/process.h
2018-09-04 19:51:10 +09:00
Dominique Martinet
ab8fe0bbbf mcctrl: convert perf ctrl ioctls to mcctrl_ihk_send_wait
While we are here, also optimize code a bit: perf_desc does not need
to be allocated for every cpu; and fix coding style.

Change-Id: Iad19fed08205d38594fd3f1b7ddf2b19a9cf0d9d
2018-07-02 16:11:01 +09:00
Dominique Martinet
b939ca9370 mcctrl: refactor prepare_image into new generic ikc send&wait
Many ikc messages expecting a reply use wait_event_interruptible
incorrectly, freeing memory that could still be used on the other side.

This commit implements a generic ikc send and wait helper that helps
with memory management and ownership properly:
 - if the message succeeds and a reply comes back normally, the memory
is freed by the caller as usual
 - if the wait fails (signal before the reply comes or timeout) then the
memory is set as owner by ikc and will be free when the reply comes back
later
 - if the reply never comes, the memory is freed at shutdown when
destroying ikc channels

Refs: #1076
Change-Id: I7f348d9029a6ad56ba9a50c836105ec39fa14943
2018-07-02 04:34:44 +00:00
Balazs Gerofi
d5629606c5 mcexec: -m: interpret as numactl -m (i.e., MPOL_BIND)
Conflicts:
	executer/include/uprotocol.h
	executer/user/mcexec.c
	kernel/include/syscall.h
2017-10-18 16:54:34 +09:00
Masamichi Takagi
6b60dee890 ihklib: Fix ihklib_rusage.h for x86 2017-10-04 05:06:17 +09:00
Masamichi Takagi
be4d84c0c1 mcexec: Add --stack-premap=<premap_size>[,<max>]
<premap_size> of stack is pre-mapped on creating a process.
And its max size of stack is set to <max>.
This replaces MCKERNEL_RLIMIT_STACK=<premap_size>,<max>.
2017-09-26 17:04:10 +09:00
Masamichi Takagi
aa7cb970c4 ihk_os_getrusage(): Compile LWK-specific results in mcctrl
1. User asks mcctrl for the result via ihk_os_getrusage() with passing void *
2. mcctrl compiles the results and passes them to the user
3. User interprets it by using the type defined in the LWK-specific header
2017-09-20 15:03:45 +09:00
Katsuya Horigome
a05b6e1ba8 Expand dump-functions for excluding user/unused memory (This is rebase commit for merging to development) 2017-09-11 15:49:04 +09:00
Ken Sato
b53fb5f5cb qlmpi: export qlmpilib.h 2017-08-30 10:37:36 +09:00
Yutaka Ishikawa
236a072311 Add qlmpi and swap to mckernel (This is rebase commit for merging to development) 2017-08-29 15:04:58 +09:00
Tomoki Shirasawa
b39fec1104 uti: remove unused functions 2017-07-26 13:14:30 +09:00
Tomoki Shirasawa
effde241b9 support uti_attr for utility thread offloading 2017-07-25 13:03:48 +09:00
Balazs Gerofi
bc423255d9 mcctrl/mcexec: limit thread pool size when too many threads exist on Linux 2017-07-21 15:33:19 +09:00
Tomoki Shirasawa
4f9865cc8f clean up unused code 2017-06-27 13:46:38 +09:00
Tomoki Shirasawa
07efb3ab9a support to utility thread offloading 2017-06-27 13:27:09 +09:00
Balazs Gerofi
200fe9aec4 mcctrl/mcexec: fix per-process data reference counting 2017-06-07 17:21:55 +09:00
Balazs Gerofi
9b5ccb5a33 Pre-map file mappings from /dev/shm (--mpol-shm-premap mcexec argument) 2017-05-23 20:00:06 +09:00
Balazs Gerofi
e901d42fb6 mcexec: --extend-heap-by: argument to specify heap extension size 2017-05-23 19:58:49 +09:00
Balazs Gerofi
e3e0f6a174 mcexec: introduction of --profile 2017-05-23 02:42:06 +09:00
Balazs Gerofi
9c25d47d9b mcexec: transfer job information to LWK 2017-05-23 02:42:06 +09:00
Balazs Gerofi
2d72042021 mcexec: bind to CPus according to ikc_map 2017-05-23 02:39:42 +09:00
Balazs Gerofi
26b9484bae mcexec: --mpol-threshold to control MPOL_BIND/MPOL_PREFERRED 2017-05-23 02:39:42 +09:00
Balazs Gerofi
182202523e mcexec/mm: user memory policy control for heap, stack, etc. 2017-05-23 02:39:42 +09:00
Ken Sato
1541b26086 ihklib: add pa_info functions. 2017-04-27 17:13:49 +09:00
Balazs Gerofi
1bc3218fc1 partitioned execution: bind mcexec to corresponding NUMA node 2017-01-30 07:24:09 +09:00
Balazs Gerofi
052b3f44ca mcexec: -n: topology aware partitioned execution 2016-12-10 16:27:57 +09:00
Balazs Gerofi
fdcf766337 prepare_process(): pass cpu_set in program_load_desc 2016-12-09 16:32:20 +09:00
Balazs Gerofi
69a5c53074 NUMA: hide non-existing nodes from /sys/devices/system/node listing 2016-11-05 16:12:08 +09:00
Balazs Gerofi
e46f027894 mcexec/mcctrl: unmount cgroups (privately) which expose invalid NUMA info 2016-11-04 17:02:48 +09:00
Balazs Gerofi
5e9957da0f syscall_response: introduction of req_thread_status field 2016-08-16 08:53:41 +09:00
Balazs Gerofi
fb84d4ef11 mcctrl: thread pool based system call offload handling 2016-08-08 19:43:05 +09:00
NAKAMURA Gou
41bb2ab5e6 support vdso which borrows clocksource from linux 2016-03-28 22:57:44 +09:00
Yoichi Umezawa
f214ff1b57 mcctrl: add MCEXEC_UP_SYS_MOUNT, MCEXEC_UP_SYS_UNSHARE 2016-02-08 16:00:52 +09:00
Tomoki Shirasawa
bb81f84709 support PIE executable for PVAS 2015-12-14 11:05:28 +09:00
Tomoki Shirasawa
59ee251e1c fix /proc/pid/mem, /proc/pid/status, /proc/pid/cmdline 2015-07-02 00:22:35 +09:00