/munetakaJupyterHub/Computer/MongoDB/CompileFromSource/compileMongoDbDebianBookworm.md

§2023-07-23

2023-08-16
- hc4Bookworm , failed

-llzma -llzma -lm -lresolv -lcrypto -lssl -lcrypto -ldl -lrt -ldl
g++ @/tmp/tmp3skv9lad.lnk
collect2: fatal error: ld terminated with signal 9 [Killed]
compilation terminated.
scons: *** [build/opt/mongo/db/mongod] Error 1
scons: building terminated because of errors.
build/opt/mongo/db/mongod failed: Error 1

on Odroid-n2+, Debian Bookworm

§2023-07-22

on Odroid-hc4 using Debian Bookworm

§2023-08-11 18:25 started, Fished 2023-08-13 12:30

on Odroid0hc4 Debian Bookworm
- git checkout r7.1.0-alpha -b test

§2023-08-16

on Odroid-n2+, Debian Bookworm
$ git checkout r7.1.0-alpha -b test
started 2023-08-16 13:00
failed with out of memory error
add more swap (32G + 8G) and restarted 2023-08-16 20:00
Still failed with

  Internal Error: no cycle found for node build/opt/mongo/s/commands/cluster_create_indexes_cmd.o (<SCons.Node.FS.File object at 0x28f8b50>) in state pending
  Internal Error: no cycle found for node build/opt/mongo/scripting/mozjs/dbpointer.o (<SCons.Node.FS.File object at 0x288c3c0>) in state pending

File "/opt/xfs/home/alexlai/build/mongo/src/third_party/scons-3.1.2/scons-local-3.1.2/SCons/Taskmaster.py", line 1076, in cleanup
build/opt/mongo/db/commands/kill_operations_gen.cpp failed: Error 1
build/opt/mongo/util/processinfo_linux.o failed: Error 1
build/opt/mongo/s/commands/cluster_abort_transaction_cmd_s.o failed: Error 1
build/opt/mongo/s/commands/cluster_write_without_shard_key_cmd.o failed: Error 1
build/opt/mongo/s/commands/cluster_query_without_shard_key_cmd.o failed: Error 1
build/opt/mongo/s/commands/cluster_map_reduce_agg.o failed: Error 1

in sudo journal |grep error a lot of /dev/sda I/O error
take /dev/sda1 out and increased swap memory with a USB sticker 32G
finished sucessfully

real    3297m05.874s
user    2803m34.731s
sys      457m45.856s

requirements

$ sudo apt update && sudo apt -y upgrade
$ sudo apt install -y build-essential libcurl4-openssl-dev liblzma-dev libssl-dev python3-dev git openssl-devel?

Prepare Pyhton3

$ puthon3 -version
3.11.2
$ sudo apt install python3.11-venv
$ python3 -m venv ~/PYTHON-3.11.2
alexlai@hc4Bullseye:~$ source ~/PYTHON-3.11.2/bin/activate
(PYTHON-3.11.2) alexlai@hc4Bullseye:~$ which pip3
/home/alexlai/PYTHON-3.11.2/bin/pip3
(PYTHON-3.11.2) alexlai@hc4Bullseye:~$ pip3 cache purge

scripts/buildMongo.sh

#!/bin/bash

# -j number of CPUS, no plus 2, odroid-n2 -j 6, odroid-hc4 -j4, orangePi5 -j 8
# execute form git clone https://github.com/mongodb/mongo.git && cd mongo
# under python3 eviromental control
# git checkout r7.0.0-rc8 -b test
# time python3 buildscripts/scons.py install-core -j 4 --ssl --linker=gold CFLAGS="-march=armv8-a+crc -mtune=generic" --disable-warnings-as-errors >log 2>&1 &

time python3 buildscripts/scons.py install-core -j4 --ssl --linker=gold CFLAGS="-march=armv8-a+crc -mtune='cortex-a73.cortex-a53' " --disable-warnings-as-errors >log 2>&1 &

#
#   install-mongod
#   install-mongos
#   install-core (includes only mongod and mongos)
#   install-servers (includes all server components)
#   install-devcore (includes mongod, mongos, and jstestshell (formerly mongo shell))
#   install-all (includes a complete end-user distribution and tests)
#    install-all-meta (absolutely everything that can be built and installed)
#

git clone

$ git clone https://github.com/mongodb/mongo.git && cd mongo
$ git checkout r7.0.0-rc8 -b test

pip requirements

$ pip3 install -r etc/pip/compile-requirements.txt
...
Successfully installed Cheetah3-3.2.6.post1 PyYAML-6.0 anyio-3.7.1 boto3-1.28.9 botocore-1.31.9 certifi-2023.5.7 
cffi-1.15.1 charset-normalizer-2.0.12 cryptography-36.0.2 distro-1.8.0 dnspython-2.4.0 gitdb-4.0.10 gitpython-3.1.32 
h11-0.14.0 httpcore-0.17.3 idna-3.4 jmespath-1.0.1 mongo-ninja-python-1.11.1.5 mongo-tooling-metrics-1.0.8 
packaging-21.3 psutil-5.8.0 pycparser-2.21 pydantic-1.10.11 pymongo-4.3.3 pymongo-auth-aws-1.1.0 pyparsing-3.1.0 
python-dateutil-2.8.2 regex-2021.11.10 requests-2.26.0 requirements_parser-0.3.1 s3transfer-0.6.1 
setuptools-58.5.3 six-1.16.0 smmap-5.0.0 sniffio-1.3.0 types-PyYAML-6.0.12.11 types-setuptools-57.4.18 
typing-extensions-4.7.1 urllib3-1.26.16

§2023-07-20 !!! Failed !!!

If you compiled python from source, then python -m venv ~/PYTHON-x.y.z and pip install -r etc/pip/compile-requirements.txt will fail.

On Debian, please use

apt install sudo apt install python3.11-venv
python3 -m venv ~/PYTHON-3.11.2
source ~/PYTHON-3.11.2/bin/activate
pip install -r etc/pip/compile-requirements.txt

¶ Compile

git clone https://github.com/mongodb/mongo.git && cd mongo
(PYTHON-3.11.2) alexlai@org5Bookworm:~/build/mongo$ git checkout r7.0.0-rc8 -b test
(PYTHON-3.11.2) alexlai@org5Bookworm:~/build/mongo$ pip install -r etc/pip/compile-requirements.txt
...
cessfully installed Cheetah3-3.2.6.post1 PyYAML-6.0 anyio-3.7.1 boto3-1.28.8 botocore-1.31.8 certifi-2023.5.7 
cffi-1.15.1 charset-normalizer-2.0.12 cryptography-36.0.2 distro-1.8.0 dnspython-2.4.0 gitdb-4.0.10 
gitpython-3.1.32 h11-0.14.0 httpcore-0.17.3 idna-3.4 jmespath-1.0.1 mongo-ninja-python-1.11.1.5 mongo-tooling-metrics-1.0.8 packaging-21.3 psutil-5.8.0 pycparser-2.21 pydantic-1.10.11 pymongo-4.3.3 pymongo-auth-aws-1.1.0 pyparsing-3.1.0 python-dateutil-2.8.2 
regex-2021.11.10 requests-2.26.0 requirements_parser-0.3.1 s3transfer-0.6.1 setuptools-58.5.3 six-1.16.0 
smmap-5.0.0 sniffio-1.3.0 types-PyYAML-6.0.12.11 types-setuptools-57.4.18 typing-extensions-4.7.1 urllib3-1.26.16

(PYTHON-3.11.2) alexlai@org5Bookworm:~/build/mongo$ time python3 buildscripts/scons.py install-core -j 8 --ssl --linker=gold CFLAGS="-march=armv8-a+crc -mtune=generic" --disable-warnings-as-errors
...
src/mongo/util/summation.h: In member function 'mongo::DoubleDouble mongo::DoubleDoubleSummation::getDoubleDouble() const':
src/mongo/util/summation.h:94:42: note: parameter passing for argument of type 'std::pair<double, double>' when C++17 is enabled changed to match C++14 in GCC 10.1
   94 |     DoubleDouble getDoubleDouble() const {
      |                                          ^
g++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
scons: *** [build/opt/mongo/db/s/migration_util.o] Error 1
scons: building terminated because of errors.
build/opt/mongo/db/s/migration_util.o failed: Error 1

real	106m50.310s
user	431m9.320s
sys	    180m59.983s

(PYTHON-3.11.2) alexlai@org5Bookworm:~/build/mongo$ git checkout r6.0.8 -b test
(PYTHON-3.11.2) alexlai@org5Bookworm:~/build/mongo$ git branch
  master
* test
(PYTHON-3.11.2) alexlai@org5Bookworm:~/build/mongo$ deactivate 
alexlai@org5Bookworm:~/build/mongo$ cd
alexlai@org5Bookworm:~$ rm -rf PYTHON-3.11.2
alexlai@org5Bookworm:~$ python3 -m venv ~/PYTHON-3.11.2
alexlai@org5Bookworm:~$ source ~/PYTHON-3.11.2/bin/activate
$ pip3 cache purge
(PYTHON-3.11.2) alexlai@org5Bookworm:~$ cd build/mongo/
(PYTHON-3.11.2) alexlai@org5Bookworm:~/build/mongo$ git branch
  master
* test
(PYTHON-3.11.2) alexlai@org5Bookworm:~/build/mongo$ python3 -m pip install -r etc/pip/compile-requirements.txt
(PYTHON-3.11.2) alexlai@org5Bookworm:~/build/mongo$ time python3 buildscripts/scons.py install-core -j 8 --ssl --linker=gold CFLAGS="-march=armv8-a+crc -mtune=generic" --disable-warnings-as-errors

...
g++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
scons: *** [build/opt/mongo/db/mongod_main.o] Error 1
scons: building terminated because of errors.

scons: *** Found dependency cycle(s):
  Internal Error: no cycle found for node build/opt/mongo/s/libquery_analysis_sampler.a 
  (<SCons.Node.FS.File object at 0x319ad410>) in state pending
  Internal Error: no cycle found for node build/opt/mongo/db/query/optimizer/props.o (<SCons.Node.FS.File object at 0x31dda010>) in state pending
  Internal Error: no cycle found for node build/opt/mongo/db/pipeline/document_source_coll_stats_gen.o (<SCons.Node.FS.File object at 0x31b58030>) in state pending
  Internal Error: no cycle found for node build/opt/mongo/util/allocator.o (<SCons.Node.FS.File object at 0x32818050>) in state pending
  Internal Error: no cycle found for node build/opt/mongo/client/replica_set_monitor_server_parameters_gen.o (<SCons.Node.FS.File object at 0x31840080>) in state pending
  Internal Error: no cycle found for node build/opt/third_party/mozjs/extract/js/src/gc/StoreBuffer.o (<SCons.Node.FS.File object at 0x312520b0>) in state pending
  Internal Error: no cycle found for node build/opt/mongo/db/pipeline/libdocument_sources_idl.a (<SCons.Node.FS.File object at 0x31c30510>) in state pending
  Internal Error: no cycle found for node build/opt/mongo/platform/process_id.o (<SCons.Node.FS.File object at 0x32530130>) in state pending
  Internal Error: no cycle found for node build/opt/mongo/db/query/optimizer/liboptimizer_base.a (<SCons.Node.FS.File object at 0x31ab1d90>) in state pending
  Internal Error: no cycle found for node build/opt/mongo/client/libreplica_set_monitor_server_parameters.a (<SCons.Node.FS.File object at 0x3183cd90>) in state pending
  Internal Error: no cycle found for node build/opt/mongo/bson/bsonelement.o (<SCons.Node.FS.File object at 0x32800210>) in state pending
  Internal Error: no cycle found for node build/opt/mongo/db/commands/fail_point_cmd.o (<SCons.Node.FS.File object at 0x317d8220>) in state pending
  Internal Error: no cycle found for node build/opt/mongo/db/pipeline/libaccumulator.a (<SCons.Node.FS.File object at 0x31c39630>) in state pending
  Internal Error: no cycle found for node build/opt/mongo/s/libsharding_api.a (<SCons.Node.FS.File object at 0x31a0c270>) in state pending
  Internal Error: no cycle found for node build/opt/mongo/db/pipeline/window_function/window_function_integral.o (<SCons.Node.FS.File object at 0x31c38330>) in state pending
  Internal Error: no cycle found for node build/opt/mongo/s/query_analysis_sampler.o (<SCons.Node.FS.File object at 0x32520370>) in state pending
  Internal Error: no cycle found for node build/opt/mongo/db/storage/storage_engine_init.o (<SCons.Node.FS.File object at 0x321c13a0>) in state pending
  Internal Error: no cycle found for node build/opt/mongo/s/cluster_ddl.o (<SCons.Node.FS.File object at 0x32470bf0>) in state pending

File "/opt/xfs/home/alexlai/build/mongo/src/third_party/scons-3.1.2/scons-local-3.1.2/SCons/Taskmaster.py", line 1076, in cleanup
build/opt/mongo/db/mongod_main.o failed: Error 1

real	247m47.379s
user	1360m6.769s
sys	243m52.989s