macOS 下编译集成 Python 3 的 GDB

作者 Known Rabbit 日期 2017-09-12
macOS 下编译集成 Python 3 的 GDB

2017-10-31更新:brew维护者明确表示需要上游合并,而从文末来看明显不可能合并了。我的PR在那放着也不会丢,从这个链接下载patch并自行apply即可。

Python是信仰。版本也是信仰的一部分。

开玩笑的啦。之所以非要自己编译,是因为gdb的某个插件不支持Py2.x。更何况,3比2在某些细节上更合我的口味。而brew的formula只支持可选链接brew版的python2,并没有3,只好自己研究一下咯。

其实让我最惊讶的是,主流发行版(如Ubuntu)有的已经把gdb链接到python3了,然而都几年了没人研究也没人提,网上也找不到——什么时候让mac也可以这么diao呀?

撰写本文时所使用的版本是GDB 8.0.1

查错

不想看可以直接跳过到下一章节如何修复报错并成功编译。

先用brew edit gdb 查看了一下formula,发现通过它指定--with-python=/usr/local/bin/python实现链接到brew的python,我试着在后面加了个3。报这样的错误。

checking whether to use python... /usr/local/opt/python3
checking for python2.7... no
checking for python2.6... no
checking for python2.5... no
checking for python2.4... no
configure: error: no usable python found at /usr/local/opt/python3
make[1]: *** [configure-gdb] Error 1
make: *** [all] Error 2

告诉我说,py3不可用。这个报错是在configure的过程中产生的,而configure是用autoconf[1]生成的。于是我就在configure.ac脚本里搜 “no usable python”,定位到如下位置。

if test "${have_libpython}" = no; then
case "${with_python}" in
yes)
AC_MSG_ERROR([python is missing or unusable])
;;
auto)
AC_MSG_WARN([python is missing or unusable; some features may be unavailable.])
;;
*)
AC_MSG_ERROR([no usable python found at ${with_python}])
;;
esac

既然with_python是我们刚指定的参数,那么have_libpython就是系统检测的可用python路径咯。

再往上翻一点,发现设置这个变量的位置在895行,作为测试从python-config获得的编译参数是否可用的测试结果,

case "${python_version}" in
python*)
AC_TRY_LIBPYTHON(${python_version}, have_libpython,
${python_includes}, ${python_libs})
;;

而编译参数的来源位于configure.ac的838行,

python_includes=`${python_prog} ${srcdir}/python/python-config.py --includes`
python_libs=`${python_prog} ${srcdir}/python/python-config.py --ldflags`
python_prefix=`${python_prog} ${srcdir}/python/python-config.py --exec-prefix`

把它的结果和原装flags比较一下,还是有一点不一样的。

$ python3 python/python-config.py --ldflags
-L/usr/local/opt/python3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/config-3.6m-darwin -ldl -framework CoreFoundation -lpython3.6m -Wl,-stack_size,1000000 -framework CoreFoundation Python.framework/Versions/3.6/Python

$ python3-config --ldflags
-L/usr/local/opt/python3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/config-3.6m-darwin -lpython3.6m -ldl -framework CoreFoundation
# 下面是ubuntu的输出
~$ python3-config --ldflags
-L/usr/lib/python3.5/config-3.5m-x86_64-linux-gnu -L/usr/lib -lpython3.5m -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions

强行把这个python-config换成了python自带的,编译通过,就这么给gdb的formula交了个PR。后来又对比一下gdb自己写的和python官方的python-config,除了细节大部分都大同小异。只有一句话。

@@ -72,7 +72,7 @@ for opt in opt_flags:
libs.insert(0, '-L' + getvar('LIBPL'))
elif os.name == 'nt':
libs.insert(0, '-L' + sysconfig.PREFIX + '/libs')
- if getvar('LINKFORSHARED') is not None:
+ if not getvar('PYTHONFRAMEWORK'):
libs.extend(getvar('LINKFORSHARED').split())
print (to_unix_path(' '.join(libs)))

在网上还有一个类似[2]的patch。直接把这10和11行删掉了。不按官方的来,我觉得不妥。

溯源

其实我更感兴趣的是,为啥实现同样的功能非要两份冗余的代码?为啥不信任官方的工具呢?我早就听说,许多发行版早就发布了集成python3的gdb。brew维护者说:要等上游更新。然而这么多年了,默认还是python2。

于是我从ppa下来ubuntu版gdb的源码研究,发现它除了src外,还有个文件夹patch。里面赫然有一个python-config.patch,写法与我的patch大同小异。(惊!历史的开拓者!)我又往下翻changelog[3],看是哪个版本加入的python3的支持。

+  * debian/patches: Update for new upstream version
+ - Remove patches subsumed by new upstream version/tarball:
+ + ppc64le.diff
+ + hurd-new-RPC-reply-stub-functions.patch
+ + hurd-adapt-to-changed-MIG-output.patch
+ + hurd-adjust-to-startup-with-shell-changes.patch
+ + hurd-make-MIG-output-parsing-more-robust.patch
+ + kfreebsd_bug752390.diff
+ + restore-run.1.patch
+ - Adjust patches to work with new upstream version:
+ + gdb-fortran-main.patch: Update for upstream changes
+ + python-config.patch: Remove parts applied upstream
+
+ [ Héctor Orón Martínez ]
+ * Switch gdb to python3 (Closes: #752581)
+ + Add gdb-python2 package for use with python2-only scripts
+ * Enable babeltrace on supported arches
+
+ [ Samuel Bronson ]
+ * Make gdb-python2 provide gdb.
+ * Totally rewrite the NEWS entry about the Python changes.
+ + And split out most of the new text to README.python_switch.
+
+ -- Samuel Bronson <naesten@gmail.com> Tue, 26 Aug 2014 15:34:03 -0400
+
gdb (7.8-0ubuntu2) utopic; urgency=medium

版本7.8-0ubuntu2。备注还是:修改了被上游合并的patch。可上游合并了为啥还是.patch呢?一看开头:debian/patches。😯……敢情你们浓眉大眼的,代码还是三手的?

再去看debian的代码,它们专门有个Debpatches服务,从那里可以找到所有debian贡献者们写的patch。所以说嘛,linux老大不是什么ubuntu,最后不还是得看debian脸色?在这里我只找到了几个release版本所含的patch,最早加入这个补丁的版本是7.6.2-1.1(jessie),可以看到时间:2013-08-14。比Ubuntu还早了一年。看来Ubuntu也不是很快很bleeding-edge嘛!

那这些年gdb开发者们都干啥吃的?其实他们知道的比香港记者都快。早在2012年12月,圣诞节前五天,就有一个ubuntu的开发者在邮件列表里为gdb-7.5提交了类似的patch。然鹅!为啥这样都没有被合并呢?看看邮件列表[4]里的对话:

From: Matthias Klose <doko at ubuntu dot com>
Date: Thu, 20 Dec 2012 17:13:14 +0100

Am 20.12.2012 16:59, schrieb Joel Brobecker:
> Can you tell us what problem you are trying to solve?

cross building in a multiarch environment, where the host libraries are
installed, and a <host>-python-config is shipped as part of the system python.
This <host>-python-config is a backport of http://bugs.python.org/issue16235
(not yet applied upstream).

==========
From: Joel Brobecker <brobecker at adacore dot com>
Date: Fri, 21 Dec 2012 11:57:21 +0400
I think that this was the plan, yes; but Doug Evans would probably
know more, as he is the main author of this code (I only fiddled
with it).

==========
From: Doug Evans <dje at google dot com>
Date: Fri, 21 Dec 2012 09:30:47 -0800

The patch is fine with me.
[The plan wasn't to shift and call the "real" python-config ... at the
time there was no real python-config for cross builds.]

==========
From: Joel Brobecker <brobecker at adacore dot com>
Date: Fri, 21 Dec 2012 21:50:59 +0400
Can we make this new behavior optional? The "real" python-config
is broken for several platforms, and having our own that we can
control was very convenient.

==========
From: Doug Evans <dje at google dot com>
Date: Sat, 22 Dec 2012 11:33:08 -0800
I certainly don't want to break your use-case.

I need to look at the patch again (but I probably won't get to it
until next week).

他缩,等老子周一上班再看。注意时间:Sat, 22 Dec。嗯,周六。我掐指一算,23号,周日。24!25!26!圣诞节,三天,小,长,假!

于是在一片欢乐祥和的气氛中,bug就这么被遗忘了(´・_・`)

修复

对了开头好像有一句没说……本文遇到的bug及修复都仅针对mac系统(亲测10.12.6),如果你一直都用apt-get装软件的话呢,它已经帮你把屁股擦干净了。更何况,mac的python-config是一个python脚本,而ubuntu下的是一个shell脚本,代码生成的,所以即便真的要改的话也不是一个方案啦。

将下面的patch放到一个文本文件(比如py.patch)里,然后把这个文件放到gdb的源码目录中,运行git apply py.patch。然后正常编译即可。这个文件已经好多版本没改啦,所以大概都能用。

diff --git a/gdb/python/python-config.py b/gdb/python/python-config.py
index c2b2969..554dac9 100644
--- a/gdb/python/python-config.py
+++ b/gdb/python/python-config.py
@@ -72,7 +72,7 @@ for opt in opt_flags:
libs.insert(0, '-L' + getvar('LIBPL'))
elif os.name == 'nt':
libs.insert(0, '-L' + sysconfig.PREFIX + '/libs')
- if getvar('LINKFORSHARED') is not None:
+ if not getvar('PYTHONFRAMEWORK'):
libs.extend(getvar('LINKFORSHARED').split())
print (to_unix_path(' '.join(libs)))

参考资料


  1. autoconf automake tutorial

  2. For gdb to execute python3 script on OSX 10.10 or later

  3. gdb_7.8-0ubuntu2_7.8-1ubuntu1.diff and Change log for gdb package in Ubuntu

  4. [patch] use python-config if available