Fix #134, FreeBSD Support, additional introspections by tony · Pull Request #136 · python-cmake-buildsystem/python-cmake-buildsystem

tony · 2016-06-12T10:47:48Z

Fix HAVE_SYS_SYS_DOMAIN_H (sys/sys_domain.h, OS X system)
setpgrp introspection
Convert cmake/fficonfig.h → cmake/fficonfig.h.in

Tested on

FreeBSD 11-CURRENT x86_64, CMake 3.5.2, clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM 3.8.0)
OS X 10.11.5 x86_64 CMake 3.5.2, Apple LLVM version 7.3.0 (clang-703.0.31)

tony · 2016-06-14T05:34:15Z

Greets @jcfr am I on the right track with this?

jcfr · 2016-06-14T05:58:09Z

cmake/ConfigureChecks.cmake

 check_include_files(sys/sendfile.h HAVE_SYS_SENDFILE_H)
 check_include_files(sys/syscall.h HAVE_SYS_SYSCALL_H)
-check_include_files(sys/domain.h HAVE_SYS_SYS_DOMAIN_H)
+check_include_files(sys/sys_domain.h HAVE_SYS_SYS_DOMAIN_H)


Good catch.

This commit has been integrated into master as 6f3f1ec

jcfr · 2016-06-17T00:22:43Z

cmake/ConfigureChecks.cmake

@@ -1765,6 +1851,8 @@ endif()

 check_c_source_compiles("#include <unistd.h>\n int main() {getpgrp(0);}" GETPGRP_HAVE_ARG)



# libffi specific

libtool isn't used.

tony · 2016-06-17T03:19:14Z

@jcfr how's this?

jcfr · 2016-06-17T03:26:11Z

cmake/fficonfig.h.in


 /* Define to the full name and version of this package. */
-#define PACKAGE_STRING "libffi 3.0.10rc0"
+#cmakedefine PACKAGE_STRING "libffi 3.0.10rc0"


Looks like this one got forgotten.

I will take care of it while integrating.

jcfr · 2016-06-17T03:30:02Z

@tony Looks good. Outstanding contributions. I will validate on MacOSX and Linux and then integrate.

On the FreeBSD side, are all the tests passing ?

(If you run ctest in the build dir )

jcfr · 2016-06-17T03:31:09Z

then integrate.

And while I will probably squash some commits together, I will of course make sure to keep the authorship.

tony · 2016-06-17T04:21:50Z

@jcfr Yes, I'm testing through a project I'm integrating it into on both:

FreeBSD 11-CURRENT x86_64, CMake 3.5.2, clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM 3.8.0) with Python 3.5
OS X 10.11.5 x86_64 CMake 3.5.2, Apple LLVM version 7.3.0 (clang-703.0.31) with Python 3.5

before every commit. I'll do ctest now also.

Also, I can squish, unless you got a plan (i'm guessing you'd like cffi-related stuff clumped/ordered together in some fashion)

prevents extensions with REQUIRES LINUX such as linuxaudiodev being added to FreeBSD builds

jcfr · 2016-06-17T04:56:08Z

CMakeLists.txt


 # Useful additional variables that extensions can use.
-if(UNIX AND NOT APPLE)
+if(CMAKE_SYSTEM MATCHES Linux)


Nice.

Should this other use of UNIX AND NOT APPLE be tweaked too ?

See

python-cmake-buildsystem/cmake/libpython/CMakeLists.txt

Line 535 in 7812f43

if(IS_PY3 AND UNIX AND NOT APPLE)

Note: This commit has been integrated as 06189e5

tony · 2016-06-17T04:56:37Z

Preliminary results. I have to read into them more

FreeBSD

99% tests passed, 3 tests failed out of 392

Total Test time (real) = 318.12 sec

The following tests FAILED:
        179 - test_ioctl (Failed)
        191 - test__locale (Failed)
        261 - test_pty (Failed)
Errors while running CTest

out of pty devices, and

191/392 Test: test__locale
Command: "/home/x/study/cmake/python-cmake-buildsystem/build/bin/python" "-Wd" "-3" "-E" "-tt" "/home/x/study/cmake/python-cmake-buildsystem/build/lib/python2.7/test/regrtest.py" "-l" "-v" "test__locale"
Directory: /usr/home/x/study/cmake/python-cmake-buildsystem/build
"test__locale" start time: Jun 16 23:39 CDT
Output:           
----------------------------------------------------------
== CPython 2.7.11 (default, Jun 16 2016, 23:35:26) [GCC 4.2.1 Compatible FreeBSD Clang 3.8.0 (tags/RELEASE_380/final 262564)]
==   FreeBSD-11.0-ALPHA3-amd64-64bit-ELF little-endian
==   /usr/home/x/study/cmake/python-cmake-buildsystem/Python-2.7.11/build/test_python_48641
Testing with flags: sys.flags(debug=0, py3k_warning=1, division_warning=1, division_new=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=1, tabcheck=2, verbose=0, unicode=0, bytes_warning=0, hash_randomization=0)
[1/1] test__locale
test_float_parsing (test.test__locale._LocaleTests) ... ok
test_lc_numeric_basic (test.test__locale._LocaleTests) ... ok
test_lc_numeric_localeconv (test.test__locale._LocaleTests) ... FAIL
test_lc_numeric_nl_langinfo (test.test__locale._LocaleTests) ... test test__locale failed -- multiple errors occurred
FAIL              

======================================================================
FAIL: test_lc_numeric_localeconv (test.test__locale._LocaleTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/x/study/cmake/python-cmake-buildsystem/build/lib/python2.7/test/test__locale.py", line 101, in test_lc_numeric_localeconv
    if self.numeric_tester('localeconv', formatting[lc], lc, loc):
  File "/home/x/study/cmake/python-cmake-buildsystem/build/lib/python2.7/test/test__locale.py", line 72, in numeric_tester
    used_locale)) 
AssertionError: Â  !=   (localeconv for thousands_sep; set to fr_FR.UTF-8, using fr_FR.UTF-8)

======================================================================
FAIL: test_lc_numeric_nl_langinfo (test.test__locale._LocaleTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/x/study/cmake/python-cmake-buildsystem/build/lib/python2.7/test/test__locale.py", line 86, in test_lc_numeric_nl_langinfo
    if self.numeric_tester('nl_langinfo', nl_langinfo(li), lc, loc):
  File "/home/x/study/cmake/python-cmake-buildsystem/build/lib/python2.7/test/test__locale.py", line 72, in numeric_tester
    used_locale)) 
AssertionError: Â  !=   (nl_langinfo for thousands_sep; set to fr_FR.UTF-8, using fr_FR.UTF-8)

----------------------------------------------------------------------

OS X

96% tests passed, 17 tests failed out of 392

Total Test time (real) = 256.91 sec

The following tests FAILED:
          6 - test___all__ (Failed)
        100 - test_distutils (Failed)
        143 - test_gdb (Failed)
        159 - test_hashlib (Failed)
        161 - test_hmac (Failed)
        187 - test_lib2to3 (Failed)
        203 - test_md5 (Failed)
        218 - test_multiprocessing (Failed)
        236 - test_pep247 (Failed)
        270 - test_random (Failed)
        288 - test_sha (Failed)
        296 - test_smtplib (Failed)
        318 - test_sundry (Failed)
        326 - test_tarfile (Failed)
        353 - test_unicodedata (Failed)
        361 - test_urllib2_localnet (Failed)
        369 - test_uuid (Failed)
Errors while running CTest

Most of these errors stem from a problem with md5.

203/392 Testing: test_md5
203/392 Test: test_md5
Command: "/Users/me/study/cmake/python-cmake-buildsystem/build/bin/python" "-Wd" "-3" "-E" "-tt" "/Users/me/study/cmake/python-cmake-buildsystem/build/lib/python2.7/test/regrtest.py" "-l" "-v" "test_md5"
Directory: /Users/me/study/cmake/python-cmake-buildsystem/build
"test_md5" start time: Jun 16 23:40 CDT
Output:
----------------------------------------------------------
ERROR:root:code for hash md5 was not found.
Traceback (most recent call last):
  File "/Users/me/study/cmake/python-cmake-buildsystem/build/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/Users/me/study/cmake/python-cmake-buildsystem/build/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type md5
ERROR:root:code for hash sha1 was not found.
Traceback (most recent call last):
  File "/Users/me/study/cmake/python-cmake-buildsystem/build/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/Users/me/study/cmake/python-cmake-buildsystem/build/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha1
ERROR:root:code for hash sha224 was not found.
Traceback (most recent call last):
  File "/Users/me/study/cmake/python-cmake-buildsystem/build/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/Users/me/study/cmake/python-cmake-buildsystem/build/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha224
ERROR:root:code for hash sha256 was not found.
Traceback (most recent call last):
  File "/Users/me/study/cmake/python-cmake-buildsystem/build/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/Users/me/study/cmake/python-cmake-buildsystem/build/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha256
ERROR:root:code for hash sha384 was not found.
Traceback (most recent call last):
  File "/Users/me/study/cmake/python-cmake-buildsystem/build/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/Users/me/study/cmake/python-cmake-buildsystem/build/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha384
ERROR:root:code for hash sha512 was not found.
Traceback (most recent call last):
  File "/Users/me/study/cmake/python-cmake-buildsystem/build/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/Users/me/study/cmake/python-cmake-buildsystem/build/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha512
== CPython 2.7.11 (default, Jun 16 2016, 23:31:44) [GCC 4.2.1 Compatible Clang 3.8.0 (tags/RELEASE_380/final)]
==   Darwin-15.5.0-x86_64-i386-64bit little-endian
==   /Users/me/study/cmake/python-cmake-buildsystem/Python-2.7.11/build/test_python_83563
Testing with flags: sys.flags(debug=0, py3k_warning=1, division_warning=1, division_new=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=1, tabcheck=2, verbose=0, unicode=0, bytes_warning=0, hash_randomization=0)
[1/1] test_md5
test test_md5 crashed -- <type 'exceptions.ImportError'>: cannot import name md5
Traceback (most recent call last):

tony · 2016-06-17T05:42:07Z

OS X: same ctest output for 7812f43 / master on and f6d0108 / freebsd-support with 2.7.11

jcfr · 2016-06-17T05:42:17Z

cmake/makefile-variables.in


-# Options to enable prebinding (for fast startup prior to Mac OS X 10.3)
-OTHER_LIBTOOL_OPT=
-


Commit integrated in ac6c28e

jcfr · 2016-06-17T08:30:59Z

@tony Remaining commits have been squashed with few tweaks in this topic:

https://github.com/python-cmake-buildsystem/python-cmake-buildsystem/tree/135-improve-libffi-configuration

Look like the test for HAVE_MMAP_FILE is missing.

Additionally, configuring on Ubuntu 15.10 / 5.2.1 20151010, I get the following output associated with the introspection:

-- Performing Test HAVE_MMAP_ANON
-- Performing Test HAVE_MMAP_ANON - Failed

ERROR: This one should be set to 1

The other one are ok:

-- Looking for alloca
-- Looking for alloca - found
-- Looking for memcpy
-- Looking for memcpy - found

Great. These correspond to the values that were hardcoded in fficonfig.h

-- Checking whether assembler supports .cfi_* directives
-- Checking whether assembler supports .cfi_* directives - no

Great. These correspond to the values that were hardcoded in fficonfig.h

-- Checking whether assembler supports PC relative relocs
-- Checking whether assembler supports PC relative relocs - yes

Great. This correspond to the value that was hardcoded in fficonfig.h

-- Performing Test HAVE_HIDDEN_VISIBILITY_ATTRIBUTE
-- Performing Test HAVE_HIDDEN_VISIBILITY_ATTRIBUTE - Success

Great. This correspond to the value that was hardcoded in fficonfig.h

-- Performing Test HAVE_MMAP_DEV_ZERO
-- Performing Test HAVE_MMAP_DEV_ZERO - Success

Great. This correspond to the value that was hardcoded in fficonfig.h

-- Performing Test SETPGRP_HAVE_ARG
-- Performing Test SETPGRP_HAVE_ARG - Failed

Great. This correspond to the value that is set in pyconfig.h after using the autoconf based system.

jcfr · 2016-06-17T08:44:24Z

re: Performing Test HAVE_MMAP_ANON - Failed

Found the issue, there was a typo

#ifndef MAPANONYMOUS -> #ifndef MAP_ANONYMOUS

-- Performing Test HAVE_MMAP_ANON
-- Performing Test HAVE_MMAP_ANON - Success

jcfr · 2016-06-17T09:25:56Z

Look like I was wrong, this one is also miconfigured:

Checking whether assembler supports .cfi_* directives - no

tony · 2016-06-17T09:42:11Z

Checking whether assembler supports .cfi_* directives - no

Are you experiencing the same problem I have in the gist? https://gist.github.com/tony/08c2cbf23eb80a7737ea9e2829992ca7

That thing is only on m68k, no?

According to wikipedia, the last time a chip was released with that arch was 1994. https://en.wikipedia.org/wiki/Motorola_68060. Sega Genesis used it. :/ I think we can get rid of that.

What do you think?

jcfr · 2016-06-17T10:15:59Z

re: Checking whether assembler supports .cfi_* directives - no

Fixed in f9a0091

re: experiencing the same problem I have in the gist

Didn't get a chance to look at the details yet. I will make a note when I will wirk again on the cross-compilation support.

re: That thing is only on m68k, no?

I am not sure.

Look like the test for HAVE_MMAP_FILE is missing.

Fixed in ccdb7e7

next step

There are still some missing tests ... basically the one in the diff here: 86b5d80

jcfr · 2016-06-17T11:27:12Z

There are still some missing tests

Test placeholders have been added in 6e21604

jcfr · 2016-06-17T12:03:10Z

@tony Thanks again for your work on this 👍

Topic has been reorganized and integrated as 846ae61

Additional issues have also been entered as #137 #138

…config_var This commit fixes a regression introduced in 97631f2 (Add support for building Python 3.7.x) where it was incorrectly assumed that PYTHONFRAMEWORK existed for Python 3.6. It turns out that the prior value in pyconfig.h.in was introduced in python/cpython@a8f8d5b4b (bpo-29585: optimize site.py startup time (python-cmake-buildsystemGH-136)) while working toward cpython 3.7 and just before python/cpython@6b42eb176 (bpo-29585: Fix sysconfig.get_config_var("PYTHONFRAMEWORK") (GH-2483)) intoducing _PYTHONFRAMEWORK and avoiding incorrect value returned by sysconfig.get_config_var. Co-authored-by: Issam E. Maghni <[email protected]>

tony added 3 commits June 12, 2016 05:42

convert fficonfig.h to template file

de77328

add introspection for setpgrp

b08fe8c

correct HAVE_SYS_SYS_DOMAIN_H include

2dc4b16

tony force-pushed the freebsd-support branch from 387f79b to 2dc4b16 Compare June 12, 2016 10:48

jcfr reviewed Jun 14, 2016
View reviewed changes

jcfr reviewed Jun 17, 2016
View reviewed changes

tony added 2 commits June 16, 2016 19:32

Remove unused LT_OBJDIR, OTHER_LIBTOOL_OPT

28e6c50

libtool isn't used.

Test for alloca

c7af1d2

tony force-pushed the freebsd-support branch from a4c25ff to c7af1d2 Compare June 17, 2016 00:36

Add comments for libffi specific tests

4b8805e

jcfr reviewed Jun 17, 2016
View reviewed changes

Prevent BSD systems from being tagged as Linux

3d4a4b4

prevents extensions with REQUIRES LINUX such as linuxaudiodev being added to FreeBSD builds

jcfr reviewed Jun 17, 2016
View reviewed changes

Detection of BSD derivs, update libpython conditional

f6d0108

jcfr reviewed Jun 17, 2016
View reviewed changes

jcfr mentioned this pull request Jun 17, 2016

SET HAVE_ALLOCA_H not obeyed? #134

Closed

jcfr closed this Jun 17, 2016

tony deleted the freebsd-support branch August 1, 2016 10:02

jcfr mentioned this pull request Jan 17, 2022

3.7.x: Fix incorrect PYTHONFRAMEWORK value returned by sysconfig.get_config_var #302

Merged

		@@ -1765,6 +1851,8 @@ endif()

		check_c_source_compiles("#include <unistd.h>\n int main() {getpgrp(0);}" GETPGRP_HAVE_ARG)


		# Options to enable prebinding (for fast startup prior to Mac OS X 10.3)
		OTHER_LIBTOOL_OPT=

Conversation

tony commented Jun 12, 2016

Uh oh!

tony commented Jun 14, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tony commented Jun 17, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jcfr commented Jun 17, 2016

Uh oh!

jcfr commented Jun 17, 2016

Uh oh!

tony commented Jun 17, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tony commented Jun 17, 2016

FreeBSD

OS X

Uh oh!

tony commented Jun 17, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jcfr commented Jun 17, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jcfr commented Jun 17, 2016

Uh oh!

jcfr commented Jun 17, 2016

Uh oh!

tony commented Jun 17, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jcfr commented Jun 17, 2016

Uh oh!

jcfr commented Jun 17, 2016

Uh oh!

jcfr commented Jun 17, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

tony commented Jun 17, 2016 •

edited

Loading

jcfr commented Jun 17, 2016 •

edited

Loading

tony commented Jun 17, 2016 •

edited

Loading

jcfr commented Jun 17, 2016 •

edited

Loading