[[ Prebuilt ]] Ensure linux thirdparty libs are prebuilt with position-independent code by livecodeian · Pull Request #7032 · livecode/livecode

livecodeian · 2019-05-13T10:16:09Z

This patch modifies the Linux configuration settings so all static libraries are compiled using the fPIC gcc flag. This ensures the thirdparty prebuilt libraries can be used by toolchains that produce position-independent executables by default.

montegoulding · 2019-05-13T10:17:14Z

@livecode-vulcan review ok 8629f0a

livecode-vulcan · 2019-05-13T10:22:06Z

💙 review by @montegoulding ok 8629f0a

…party-PIC [[ Prebuilt ]] Ensure linux thirdparty libs are prebuilt with position-independent code This patch modifies the Linux configuration settings so all static libraries are compiled using the `fPIC` gcc flag. This ensures the thirdparty prebuilt libraries can be used by toolchains that produce position-independent executables by default.

livecode-vulcan · 2019-05-13T10:29:42Z

😞 test failure 8629f0a

try-community-armv6-android-sdk26_ndk16r15: success
try-community-armv7-android-sdk26_ndk16r15: failure
try-community-arm64-android-sdk26_ndk16r15: failure
try-community-x86-android-sdk26_ndk16r15: failure
try-community-x86_64-android-sdk26_ndk16r15: failure
try-community-js-emscripten-sdk1.35: failure
try-community-universal-ios-iphoneos12.1: failure
try-community-universal-ios-iphonesimulator12.1: failure
try-community-universal-mac-macosx10.9: failure
try-community-x86-linux-debian8: failure
try-community-x86_64-linux-debian8: failure
try-community-x86-win32: failure
try-community-x86_64-win32: failure

montegoulding · 2019-05-13T10:32:55Z

We probably shouldn't merge this until we build prebuilts against a branch of this + updated thirdparty submodule

mwieder · 2019-05-13T16:18:12Z

Yes, this appears to be failing because the prebuilt libraries aren't built yet:

curl: (22) The requested URL returned error: 400 Bad Request
Failed to find library Thirdparty-170e8a9b470c0cbea560648b3c666b5483f1f657-linux-i386-PIC either remotely or locally

…n-independent code

livecodeali · 2019-06-02T21:30:05Z

@livecode-vulcan review ok af5c40b

livecode-vulcan · 2019-06-02T21:32:22Z

💙 review by @livecodeali ok af5c40b

…party-PIC [[ Prebuilt ]] Ensure linux thirdparty libs are prebuilt with position-independent code This patch modifies the Linux configuration settings so all static libraries are compiled using the `fPIC` gcc flag. This ensures the thirdparty prebuilt libraries can be used by toolchains that produce position-independent executables by default.

livecode-vulcan · 2019-06-02T21:44:29Z

😞 test failure af5c40b

try-community-armv6-android-sdk26_ndk16r15: success
try-community-armv7-android-sdk26_ndk16r15: pending
try-community-arm64-android-sdk26_ndk16r15: pending
try-community-x86-android-sdk26_ndk16r15: pending
try-community-x86_64-android-sdk26_ndk16r15: pending
try-community-js-emscripten-sdk1.35: pending
try-community-universal-ios-iphoneos12.1: pending
try-community-universal-ios-iphonesimulator12.1: success
try-community-universal-mac-macosx10.9: pending
try-community-x86-linux-debian8: pending
try-community-x86_64-linux-debian8: pending
try-community-x86-win32: failure
try-community-x86_64-win32: failure

livecodeali · 2019-06-02T21:53:58Z

Odd... it looks as though the Windows thirdparty prebuilts didn't suffix with PIC

mwieder · 2019-06-02T21:59:14Z

Can I humbly suggest that we just merge in my (currently closed) PR #6897 for now so that linux compilations can just work and I don't have to merge that branch locally every compile time? And then replace it with Ian's patch when we're ready for it?

livecodeian · 2019-06-02T22:11:41Z

@mwieder I explained in my original comment why we can't just merge your patch and that reason still stands - it will break compilation on our Linux builders. This patch once sorted out will fix the issue so please have patience.

montegoulding · 2019-06-02T22:32:19Z

@livecodeali @livecodeian the issue is Thirdparty is not listed in lib_versions.bat so Thirdparty_BUILDREVISION is never defined

mwieder · 2019-06-03T04:02:43Z

@livecodeian Ah, you're right - I'd forgotten about that. I'm looking forward to us building on a modern linux distro some day.

livecodeali · 2019-06-03T14:47:21Z

@livecode-vulcan review ok c010244

livecode-vulcan · 2019-06-03T14:53:29Z

💙 review by @livecodeali ok c010244

…party-PIC [[ Prebuilt ]] Ensure linux thirdparty libs are prebuilt with position-independent code This patch modifies the Linux configuration settings so all static libraries are compiled using the `fPIC` gcc flag. This ensures the thirdparty prebuilt libraries can be used by toolchains that produce position-independent executables by default.

livecode-vulcan · 2019-06-03T15:44:52Z

😎 test success c010244

try-community-armv6-android-sdk26_ndk16r15: success
try-community-armv7-android-sdk26_ndk16r15: success
try-community-arm64-android-sdk26_ndk16r15: success
try-community-x86-android-sdk26_ndk16r15: success
try-community-x86_64-android-sdk26_ndk16r15: success
try-community-js-emscripten-sdk1.35: success
try-community-universal-ios-iphoneos12.1: success
try-community-universal-ios-iphonesimulator12.1: success
try-community-universal-mac-macosx10.9: success
try-community-x86-linux-debian8: success
try-community-x86_64-linux-debian8: success
try-community-x86-win32: success
try-community-x86_64-win32: success

mwieder · 2019-06-03T22:54:48Z

I can build from source now, but now I get a crash on attempting to launch.

[2387] MCU_library_load /home/mwieder/livecode/linux-x86_64-bin/dbsqlite -> /home/mwieder/livecode/linux-x86_64-bin/dbsqlite
free(): invalid pointer
Aborted (core dumped)

The top of my git log is
Merge: 3361d5f c010244
Author: Ali Lloyd [email protected]
Date: Mon Jun 3 23:07:06 2019 +0100

Merge pull request #7032 from livecodeian/feature-prebuilt-thirdparty-PIC

[[ Prebuilt ]] Ensure linux thirdparty libs are prebuilt with position-independent code

montegoulding · 2019-06-03T22:58:43Z

@mwieder is your thirdparty submodule stale?

mwieder · 2019-06-03T23:52:55Z

dunno. I did a make clean-linux before building, as always. And git-pulled all the submodules.

montegoulding · 2019-06-04T00:13:50Z

@mwieder ok, any chance you can run it under gdb and give us a backtrace?

mwieder · 2019-06-04T00:42:01Z

Here's 24 lines of backtrace... that looks pretty specific... do you need more?

#0 0x00007ffff67c7e97 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x00007ffff67c9801 in __GI_abort () at abort.c:79
#2 0x00007ffff6812897 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7ffff693fb9a "%s\n") at ../sysdeps/posix/libc_fatal.c:181
#3 0x00007ffff681990a in malloc_printerr (str=str@entry=0x7ffff693dd88 "free(): invalid pointer")
at malloc.c:5350
#4 0x00007ffff6820e1c in _int_free (have_lock=0, p=0x7fffffffa4b8, av=0x7ffff6b74c40 <main_arena>)
at malloc.c:4157
#5 0x00007ffff6820e1c in __GI___libc_free (mem=0x7fffffffa4c8) at malloc.c:3124
#6 0x00007fffeb5a9068 in sqlite3_free (p=0x2) at ../../thirdparty/libsqlite/src/sqlite3.c:26832
#7 0x00007fffeb596084 in DBConnection_SQLITE::basicExec(char const*, unsigned int*) (this=0x5555588ca3b0, q=0x555558885630 "BEGIN TRANSACTION", rows=0x7fffffffa558)
at ../../revdb/src/sqlite_connection.cpp:660
#8 0x00007fffeb595230 in DBConnection_SQLITE::sqlExecute(char*, DBString*, int, unsigned int&) (this=0x5555588ca3b0, query=0x555558885630 "BEGIN TRANSACTION", args=0x0, numargs=0, affectedrows=@0x7ff

fffffa558: 0) at ../../revdb/src/sqlite_connection.cpp:189
#9 0x00007fffedcb34e3 in REVDB_Execute(char**, int, char**, int*, int*) (p_arguments=0x5555588827e0, p_argument_count=2, p_return_string=0x7fffffffa5d8, p_pass=0x7fffffffa5d0, p_error=0x7fffffffa5d4) at ../../revdb/src/revdb.cpp:1057
#10 0x0000555555880044 in MCExternalV0::Handle(MCObject*, Handler_type, unsigned int, MCParameter*) (this=0x555557b80680, p_context=0x555556dd9870, p_type=HT_MIN, p_index=44, p_parameters=0x0)
at ../../engine/src/externalv0.cpp:342
#11 0x000055555587ee06 in MCExternalHandlerList::Handle(MCObject*, Handler_type, __MCName*, MCParameter*) (this=0x555557091240, p_object=0x555556dd9870, p_type=HT_MIN, p_message=
0x555557fc7e80, p_parameters=0x55555817d170) at ../../engine/src/external.cpp:210
#12 0x00005555557566e3 in MCStack::handle(Handler_type, __MCName*, MCParameter*, MCObject*) (this=0x555556dd9870, htype=HT_MIN, message=0x555557fc7e80, params=0x55555817d170, passing_object=0x55555815ee40) at ../../engine/src/stack.cpp:1635
#13 0x0000555555756811 in MCStack::handle(Handler_type, __MCName*, MCParameter*, MCObject*) (this=0x55555815ee40, htype=HT_MIN, message=0x555557fc7e80, params=0x55555817d170, passing_object=0x55555815

#20 0x0000555555756626 in MCStack::handle(Handler_type, __MCName*, MCParameter*, MCObject*) (this=0x55555815ee40, htype=HT_MIN, message=0x55555817c150, params=0x55555831da00, passing_object=0x0)
at ../../engine/src/stack.cpp:1617
#21 0x00005555556b40d7 in MCDispatch::handle(Handler_type, __MCName*, MCParameter*, MCObject*) (this=0x555556c54a00, htype=HT_MIN, mess=0x55555817c150, params=0x55555831da00, pass_from=0x555556dd9870) at ../../engine/src/dispatch.cpp:242
#22 0x0000555555756811 in MCStack::handle(Handler_type, __MCName*, MCParameter*, MCObject*) (this=0x555556dd9870, htype=HT_MIN, message=0x55555817c150, params=0x55555831da00, passing_object=0x555557bfd170) at ../../engine/src/stack.cpp:1661
#23 0x0000555555756811 in MCStack::handle(Handler_type, __MCName*, MCParameter*, MCObject*) (this=0x555557bfd170, htype=HT_MIN, message=0x55555817c150, params=0x55555831da00, passing_object=0x555557bfd170) at ../../engine/src/stack.cpp:1661
#24 0x0000555555971011 in MCKeywordsExecCommandOrFunction(MCExecContext&, MCHandler*, MCParameter*, __MCName*, unsigned short, unsigned short, bool, bool) (ctxt=..., handler=0x0, params=0x55555831da00, name=0x55555817c150, line=10855, pos=1, global_handler=false, is_function=false)

montegoulding · 2019-06-04T01:51:19Z

Interesting... will need to discuss with the team. It looks like sqlite_exec is returning something other than SQLITE_OK but not setting the error string.

mwieder · 2019-06-04T04:29:33Z

Let me know if you need anything further from me on this... I'm leaving town tomorrow for about three weeks.

mwieder · 2019-07-23T03:39:05Z

Still the same after pulling and rebuilding. Here's the end of the strace log:

stat("/home/mwieder/livecode/linux-x86_64-bin/sqlite.db", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
lseek(10, 0, SEEK_SET) = 0
read(10, "", 100) = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2free(): invalid pointer
) = 24
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9f03815000
rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
getpid() = 31495
gettid() = 31495
tgkill(31495, 31495, SIGABRT) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=31495, si_uid=1000} ---
+++ killed by SIGABRT (core dumped) +++
Aborted (core dumped)

montegoulding · 2019-07-23T03:56:18Z

@mwieder I think @livecodefraser found a workaround for this recently. It's due to an ABI difference between our build server and your build machine. Try adding the following to the dbsqlite and dbsqlite-server targets within the 'OS == "linux"', blocks in revdb/revdb.gyp then reconfigure:

'defines':
[
	'_GLIBCXX_USE_CXX11_ABI=0',
],

montegoulding · 2019-07-23T03:57:30Z

PS if ^ works either push up a PR for it or I will if you let me know.

mwieder · 2019-07-23T05:03:27Z

Done. #7127

livecodeian added the bug label May 13, 2019

livecodeian mentioned this pull request May 13, 2019

[[ Prebuilt ]] Build Linux prebuilt libraries with position-independant code #6953

Merged

livecode-vulcan added the vulcan/reviewed label May 13, 2019

runrevmark modified the milestones: 9.5.0-dp-1, 9.5.0-dp-2 May 13, 2019

[[ Prebuilt ]] Ensure linux thirdparty libs are prebuilt with positio…

c010244

…n-independent code

livecodeian force-pushed the feature-prebuilt-thirdparty-PIC branch from 8629f0a to af5c40b Compare May 20, 2019 01:35

livecode-vulcan removed the vulcan/reviewed label May 20, 2019

livecode-vulcan added the vulcan/reviewed label Jun 2, 2019

livecodeian force-pushed the feature-prebuilt-thirdparty-PIC branch from af5c40b to c010244 Compare June 3, 2019 10:58

livecode-vulcan removed the vulcan/reviewed label Jun 3, 2019

livecode-vulcan added the vulcan/reviewed label Jun 3, 2019

livecodeali merged commit f00d7e3 into livecode:develop Jun 3, 2019

Conversation

livecodeian commented May 13, 2019

Uh oh!

montegoulding commented May 13, 2019

Uh oh!

livecode-vulcan commented May 13, 2019

Uh oh!

livecode-vulcan commented May 13, 2019

Uh oh!

montegoulding commented May 13, 2019

Uh oh!

mwieder commented May 13, 2019

Uh oh!

livecodeali commented Jun 2, 2019

Uh oh!

livecode-vulcan commented Jun 2, 2019

Uh oh!

livecode-vulcan commented Jun 2, 2019

Uh oh!

livecodeali commented Jun 2, 2019

Uh oh!

mwieder commented Jun 2, 2019

Uh oh!

livecodeian commented Jun 2, 2019

Uh oh!

montegoulding commented Jun 2, 2019

Uh oh!

mwieder commented Jun 3, 2019

Uh oh!

livecodeali commented Jun 3, 2019

Uh oh!

livecode-vulcan commented Jun 3, 2019

Uh oh!

livecode-vulcan commented Jun 3, 2019

Uh oh!

mwieder commented Jun 3, 2019

Uh oh!

montegoulding commented Jun 3, 2019

Uh oh!

mwieder commented Jun 3, 2019

Uh oh!

montegoulding commented Jun 4, 2019

Uh oh!

mwieder commented Jun 4, 2019

Uh oh!

montegoulding commented Jun 4, 2019

Uh oh!

mwieder commented Jun 4, 2019

Uh oh!

mwieder commented Jul 23, 2019

Uh oh!

montegoulding commented Jul 23, 2019

Uh oh!

montegoulding commented Jul 23, 2019

Uh oh!

mwieder commented Jul 23, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants