[PATCH 0/4 v2] tools: add a directory to contain usefull user-facing tools

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH 0/4 v2] tools: add a directory to contain usefull user-facing tools

Yann E. MORIN-2
Hello All!

Currently, we provide a few user-facing utilities, like get-developers,
in support/scripts/ . But this directory also contains internal scripts
that a user should not be directly concerned with. Besides, it is
two-level deep in the hierarchy, which is not really nice.

So, we introduce tools/ as a top-level directory, with the goal to store
and expose all user-facing utilities, while keeping our internal scripts
in support/scripts/ .

The first patch in the series introduces a new utility, brmake, a
wrapper around make that stores all the output to a log file and
displays only the >>> lines.

The second and subsequent patches each move a few user-facing utilities
out of support/scripts/ and into tools/ .

Changes v1 -> v2:
  - rename contrib -> tools  (Arnout)
  - move more scripts


Regards,
Yann E. MORIN.


The following changes since commit 859764ac39c18c6aaabbb6a1a47f2fa2e5793044

  linux-headers: bump 4.{1, 4, 9, 11}.x series (2017-06-17 16:17:04 +0200)


are available in the git repository at:

  git://git.buildroot.org/~ymorin/git/buildroot.git

for you to fetch changes up to eb8cb9682d84d1f110044a1448f18ef95ad79e84

  tools: move check-package out of support/scripts/ (2017-06-17 21:09:20 +0200)


----------------------------------------------------------------
Yann E. MORIN (4):
      tools: add a directory to store some useful user-facing tools
      tools: move test-pkg out of support/scripts
      tools: move get-developers out of support/scripts/
      tools: move check-package out of support/scripts/

 docs/manual/adding-packages-tips.txt               |  12 +-
 docs/manual/contribute.txt                         |   2 +-
 docs/manual/writing-rules.txt                      |   5 +-
 support/scripts/check-package                      | 145 +--------------
 support/scripts/get-developers                     |  84 +--------
 support/scripts/pkg-stats                          |   2 +-
 support/scripts/test-pkg                           | 198 +--------------------
 tools/00-README.txt                                |  23 +++
 tools/brmake                                       |  37 ++++
 {support/scripts => tools}/check-package           |   2 +-
 .../scripts => tools}/checkpackagelib/__init__.py  |   0
 {support/scripts => tools}/checkpackagelib/base.py |   0
 {support/scripts => tools}/checkpackagelib/lib.py  |   0
 .../checkpackagelib/lib_config.py                  |   0
 .../scripts => tools}/checkpackagelib/lib_hash.py  |   0
 .../scripts => tools}/checkpackagelib/lib_mk.py    |   0
 .../scripts => tools}/checkpackagelib/lib_patch.py |   0
 .../scripts => tools}/checkpackagelib/readme.txt   |  10 +-
 {support/scripts => tools}/get-developers          |   0
 {support/scripts => tools}/getdeveloperlib.py      |   0
 {support/scripts => tools}/test-pkg                |   0
 21 files changed, 80 insertions(+), 440 deletions(-)
 mode change 100755 => 120000 support/scripts/check-package
 mode change 100755 => 120000 support/scripts/get-developers
 mode change 100755 => 120000 support/scripts/test-pkg
 create mode 100644 tools/00-README.txt
 create mode 100755 tools/brmake
 copy {support/scripts => tools}/check-package (98%)
 rename {support/scripts => tools}/checkpackagelib/__init__.py (100%)
 rename {support/scripts => tools}/checkpackagelib/base.py (100%)
 rename {support/scripts => tools}/checkpackagelib/lib.py (100%)
 rename {support/scripts => tools}/checkpackagelib/lib_config.py (100%)
 rename {support/scripts => tools}/checkpackagelib/lib_hash.py (100%)
 rename {support/scripts => tools}/checkpackagelib/lib_mk.py (100%)
 rename {support/scripts => tools}/checkpackagelib/lib_patch.py (100%)
 rename {support/scripts => tools}/checkpackagelib/readme.txt (91%)
 copy {support/scripts => tools}/get-developers (100%)
 rename {support/scripts => tools}/getdeveloperlib.py (100%)
 copy {support/scripts => tools}/test-pkg (100%)

--
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH 1/4 v2] tools: add a directory to store some useful user-facing tools

Yann E. MORIN-2
Add a tools directory to store useful little scripts, tools, etc...
Users may add that directory in their PATH if they want to, but this
is not mandatory.

Add the first tool: brmake, a small script that redirects the build
output log to a file, keeping just Buildroot's own messages, with the
date+time added at the start of the line.

We need to unbuffer the output of make so that, when the script is
interrupted (SIGINT, ^C), there is no lingering output not yet digested
by the logger loop.

Signed-off-by: "Yann E. MORIN" <[hidden email]>
Cc: Maxime Hadjinlian <[hidden email]>
Cc: Arnout Vandecappelle <[hidden email]>

---
Changes v1 -> v2:
  - rename contrib -> tools  (Arnout)
  - detect missing unbuffer  (Arnout)

Notes:
  - Arnout suggested using 'stdbuf' instead of 'unbuffer', but that has
    some shortcomings in weird corner-cases because it uses LD_PRELOAD
    to call a constructor that forces settings on stdout/stderr/stdin
    (it cost me three days to investigate a build failure because of
    that). So, I stuck with 'unbuffer'...
  - I also tried to use bash' coproc, but it seems there was a
    regression in later bash versions that causes an incorrect
    redirect. So, I stuck with 'unbuffer'...
---
 docs/manual/writing-rules.txt |  5 +++--
 tools/00-README.txt           |  9 +++++++++
 tools/brmake                  | 37 +++++++++++++++++++++++++++++++++++++
 3 files changed, 49 insertions(+), 2 deletions(-)
 create mode 100644 tools/00-README.txt
 create mode 100755 tools/brmake

diff --git a/docs/manual/writing-rules.txt b/docs/manual/writing-rules.txt
index 3a2c8dcb37..c144874a0d 100644
--- a/docs/manual/writing-rules.txt
+++ b/docs/manual/writing-rules.txt
@@ -146,5 +146,6 @@ syntax, refer to http://www.methods.co.nz/asciidoc/userguide.html[].
 
 === Support scripts
 
-Some scripts in the +support/+ directory are written in Python and should follow
-the https://www.python.org/dev/peps/pep-0008/[PEP8 Style Guide for Python Code].
+Some scripts in the +support/+ and +tools/+ directories are written in
+Python and should follow the
+https://www.python.org/dev/peps/pep-0008/[PEP8 Style Guide for Python Code].
diff --git a/tools/00-README.txt b/tools/00-README.txt
new file mode 100644
index 0000000000..ffc0fbf5ef
--- /dev/null
+++ b/tools/00-README.txt
@@ -0,0 +1,9 @@
+This directory contains various useful scripts and tools for working
+with Buildroot. You need not add this directory in your PATH to use
+any of those tools, but you may do so if you want.
+
+brmake
+    a script that can be run instead of make, that prepends the date in
+    front of each line, redirects all of the build output to a file
+    ("'br.log' in the current directory), and just ouputs the Buildroot
+    messages (those lines starting with >>>) on stdout.
diff --git a/tools/brmake b/tools/brmake
new file mode 100755
index 0000000000..9f7b4bc65f
--- /dev/null
+++ b/tools/brmake
@@ -0,0 +1,37 @@
+#!/bin/bash
+# (C) 2016, "Yann E. MORIN" <[hidden email]>
+# License: WTFPL, https://spdx.org/licenses/WTFPL.html
+
+main() {
+    local ret start d h m mf fs
+
+    if ! which unbuffer >/dev/null 2>&1; then
+        printf "you need to install 'unbuffer' (from package expect or expect-dev)\n" >&2
+        exit 1
+    fi
+
+    start=${SECONDS}
+
+    ( exec 2>&1; unbuffer make "${@}"; ) \
+    > >( while read line; do
+             printf "%(%Y-%m-%dT%H:%M:%S)T %s\n" -1 "${line}"
+         done \
+         |tee -a br.log \
+         |grep --colour=never -E '>>>'
+       )
+    ret=${?}
+
+    d=$((SECONDS-start))
+    printf "Done in "
+    h=$((d/3600))
+    d=$((d%3600))
+    [ ${h} -eq 0 ] || { printf "%dh " ${h}; mf="02"; }
+    m=$((d/60))
+    d=$((d%60))
+    [ ${m} -eq 0 ] || { printf "%${mf}dmin " ${m}; sf="02"; }
+    printf "%${sf}ds\n" ${d}
+
+    return ${ret}
+}
+
+main "${@}"
--
2.11.0

_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH 2/4 v2] tools: move test-pkg out of support/scripts

Yann E. MORIN-2
In reply to this post by Yann E. MORIN-2
Move it to the top-level tools/ directory, so that it is easier to
find for users.

Add a legacy symlink for those users who already used them, so as
not to break their habits.

Signed-off-by: "Yann E. MORIN" <[hidden email]>
Cc: Thomas Petazzoni <[hidden email]>
Cc: Arnout Vandecappelle <[hidden email]>
---
 docs/manual/adding-packages-tips.txt |  12 +--
 support/scripts/test-pkg             | 198 +----------------------------------
 tools/00-README.txt                  |   5 +
 tools/test-pkg                       | 197 ++++++++++++++++++++++++++++++++++
 4 files changed, 209 insertions(+), 203 deletions(-)
 mode change 100755 => 120000 support/scripts/test-pkg
 create mode 100755 tools/test-pkg

diff --git a/docs/manual/adding-packages-tips.txt b/docs/manual/adding-packages-tips.txt
index d1eab2b440..0047dfd14f 100644
--- a/docs/manual/adding-packages-tips.txt
+++ b/docs/manual/adding-packages-tips.txt
@@ -45,9 +45,9 @@ continuously test random configurations. However, these only build the
 `master` branch of the git tree, and your new fancy package is not yet
 there.
 
-Buildroot provides a script in +support/scripts/test-pkg+ that uses the
-same base configurations as used by the autobuilders so you can test
-your package in the same conditions.
+Buildroot provides a script in +tools/test-pkg+ that uses the same base
+configurations as used by the autobuilders so you can test your package
+in the same conditions.
 
 First, create a config snippet that contains all the necessary options
 needed to enable your package, but without any architecture or toolchain
@@ -74,7 +74,7 @@ Then run the +test-pkg+ script, by telling it what config snippet to use
 and what package to test:
 
 ----
-$ ./support/scripts/test-pkg -c libcurl.config -p libcurl
+$ ./tools/test-pkg -c libcurl.config -p libcurl
 ----
 
 This will try to build your package against all the toolchains used
@@ -83,7 +83,7 @@ too long to do so). The output lists all toolchains and the corresponding
 result (excerpt, results are fake):
 
 ----
-$ ./support/scripts/test-pkg -c libcurl.config -p libcurl
+$ ./tools/test-pkg -c libcurl.config -p libcurl
                 armv5-ctng-linux-gnueabi [ 1/11]: OK
               armv7-ctng-linux-gnueabihf [ 2/11]: OK
                         br-aarch64-glibc [ 3/11]: SKIPPED
@@ -124,7 +124,7 @@ The +test-pkg+ script accepts a few options, for which you can get some
 help by running:
 
 ----
-$ ./support/scripts/test-pkg -h
+$ ./tools/test-pkg -h
 ----
 
 [[github-download-url]]
diff --git a/support/scripts/test-pkg b/support/scripts/test-pkg
deleted file mode 100755
index 7633a21e53..0000000000
--- a/support/scripts/test-pkg
+++ /dev/null
@@ -1,197 +0,0 @@
-#!/usr/bin/env bash
-set -e
-
-TOOLCHAINS_URL='http://autobuild.buildroot.org/toolchains/configs/toolchain-configs.csv'
-
-main() {
-    local o O opts
-    local cfg dir pkg random toolchain
-    local ret nb nb_skip nb_fail nb_legal nb_tc build_dir
-    local -a toolchains
-
-    o='hc:d:p:r:'
-    O='help,config-snippet:build-dir:package:,random:'
-    opts="$(getopt -n "${my_name}" -o "${o}" -l "${O}" -- "${@}")"
-    eval set -- "${opts}"
-
-    random=0
-    while [ ${#} -gt 0 ]; do
-        case "${1}" in
-        (-h|--help)
-            help; exit 0
-            ;;
-        (-c|--config-snippet)
-            cfg="${2}"; shift 2
-            ;;
-        (-d|--build-dir)
-            dir="${2}"; shift 2
-            ;;
-        (-p|--package)
-            pkg="${2}"; shift 2
-            ;;
-        (-r|--random)
-            random="${2}"; shift 2
-            ;;
-        (--)
-            shift; break
-            ;;
-        esac
-    done
-    if [ -z "${cfg}" ]; then
-        printf "error: no config snippet specified\n" >&2; exit 1
-    fi
-    if [ ! -e "${cfg}" ]; then
-        printf "error: %s: no such file\n" "${cfg}" >&2; exit 1
-    fi
-    if [ -z "${dir}" ]; then
-        dir="${HOME}/br-test-pkg"
-    fi
-
-    # Extract the URLs of the toolchains; drop internal toolchains
-    # E.g.: http://server/path/to/name.config,arch,libc
-    #  -->  http://server/path/to/name.config
-    toolchains=($(curl -s "${TOOLCHAINS_URL}" \
-                  |sed -r -e 's/,.*//; /internal/d;' \
-                  |if [ ${random} -gt 0 ]; then \
-                      sort -R |head -n ${random}
-                   else
-                      cat
-                   fi |sort
-                 )
-               )
-
-    nb_tc="${#toolchains[@]}"
-    if [ ${nb_tc} -eq 0 ]; then
-        printf "error: no toolchain found (networking issue?)\n" >&2; exit 1
-    fi
-
-    nb=0
-    nb_skip=0
-    nb_fail=0
-    nb_legal=0
-    for toolchainconfig in "${toolchains[@]}"; do
-        : $((nb++))
-        # Using basename(1) on a URL works nicely
-        toolchain="$(basename "${toolchainconfig}" .config)"
-        build_dir="${dir}/${toolchain}"
-        printf "%40s [%*d/%d]: " "${toolchain}" ${#nb_tc} ${nb} ${nb_tc}
-        build_one "${build_dir}" "${toolchainconfig}" "${cfg}" "${pkg}" && ret=0 || ret=${?}
-        case ${ret} in
-        (0) printf "OK\n";;
-        (1) : $((nb_skip++)); printf "SKIPPED\n";;
-        (2) : $((nb_fail++)); printf "FAILED\n";;
-        (3) : $((nb_legal++)); printf "FAILED\n";;
-        esac
-    done
-
-    printf "%d builds, %d skipped, %d build failed, %d legal-info failed\n" \
-        ${nb} ${nb_skip} ${nb_fail} ${nb_legal}
-}
-
-build_one() {
-    local dir="${1}"
-    local url="${2}"
-    local cfg="${3}"
-    local pkg="${4}"
-
-    mkdir -p "${dir}"
-
-    if ! curl -s "${url}" >"${dir}/.config"; then
-        return 2
-    fi
-
-    cat >>"${dir}/.config" <<-_EOF_
- BR2_INIT_NONE=y
- BR2_SYSTEM_BIN_SH_NONE=y
- # BR2_PACKAGE_BUSYBOX is not set
- # BR2_TARGET_ROOTFS_TAR is not set
- _EOF_
-    cat "${cfg}" >>"${dir}/.config"
-
-    if ! make O="${dir}" olddefconfig > "${dir}/logfile" 2>&1; then
-        return 2
-    fi
-    # We want all the options from the snippet to be present as-is (set
-    # or not set) in the actual .config; if one of them is not, it means
-    # some dependency from the toolchain or arch is not available, in
-    # which case this config is untestable and we skip it.
-    # We don't care about the locale to sort in, as long as both sort are
-    # done in the same locale.
-    comm -23 <(sort "${cfg}") <(sort "${dir}/.config") >"${dir}/missing.config"
-    if [ -s "${dir}/missing.config" ]; then
-        return 1
-    fi
-    # Remove file, it's empty anyway.
-    rm -f "${dir}/missing.config"
-
-    if [ -n "${pkg}" ]; then
-        if ! make O="${dir}" "${pkg}-dirclean" >> "${dir}/logfile" 2>&1; then
-            return 2
-        fi
-    fi
-
-    # shellcheck disable=SC2086
-    if ! make O="${dir}" ${pkg} >> "${dir}/logfile" 2>&1; then
-        return 2
-    fi
-
-    # legal-info done systematically, because some packages have different
-    # sources depending on the configuration (e.g. lua-5.2 vs. lua-5.3)
-    if [ -n "${pkg}" ]; then
-        if ! make O="${dir}" "${pkg}-legal-info" >> "${dir}/logfile" 2>&1; then
-            return 3
-        fi
-    fi
-}
-
-help() {
-    cat <<_EOF_
-test-pkg: test-build a package against various toolchains and architectures
-
-The supplied config snippet is appended to each toolchain config, the
-resulting configuration is checked to ensure it still contains all options
-specified in the snippet; if any is missing, the build is skipped, on the
-assumption that the package under test requires a toolchain or architecture
-feature that is missing.
-
-In case failures are noticed, you can fix the package and just re-run the
-same command again; it will re-run the test where it failed. If you did
-specify a package (with -p), the package build dir will be removed first.
-
-The list of toolchains is retrieved from the Buildroot autobuilders, available
-at ${TOOLCHAINS_URL}.
-
-Options:
-
-    -h, --help
-        Print this help.
-
-    -c CFG, --config-snippet CFG
-        Use the CFG file as the source for the config snippet. This file
-        should contain all the config options required to build a package.
-
-    -d DIR, --build-dir DIR
-        Do the builds in directory DIR, one sub-dir per toolchain.
-
-    -p PKG, --package PKG
-        Test-build the package PKG, by running 'make PKG'; if not specified,
-        just runs 'make'.
-
-    -r N, --random N
-        Limit the tests to the N randomly selected toolchains, instead of
-        building with all toolchains.
-
-Example:
-
-    Testing libcec would require a config snippet that contains:
-        BR2_PACKAGE_LIBCEC=y
-
-    Testing libcurl with openSSL support would require a snippet such as:
-        BR2_PACKAGE_OPENSSL=y
-        BR2_PACKAGE_LIBCURL=y
-
-_EOF_
-}
-
-my_name="${0##*/}"
-main "${@}"
diff --git a/support/scripts/test-pkg b/support/scripts/test-pkg
new file mode 120000
index 0000000000..2acaa87478
--- /dev/null
+++ b/support/scripts/test-pkg
@@ -0,0 +1 @@
+../../tools/test-pkg
\ No newline at end of file
diff --git a/tools/00-README.txt b/tools/00-README.txt
index ffc0fbf5ef..092e0eac8e 100644
--- a/tools/00-README.txt
+++ b/tools/00-README.txt
@@ -7,3 +7,8 @@ brmake
     front of each line, redirects all of the build output to a file
     ("'br.log' in the current directory), and just ouputs the Buildroot
     messages (those lines starting with >>>) on stdout.
+
+test-pkg
+    a script that tests a specific package against a set of various
+    toolchains, with the goal to detect toolchain-related dependencies
+    (wchar, threads...)
diff --git a/tools/test-pkg b/tools/test-pkg
new file mode 100755
index 0000000000..7633a21e53
--- /dev/null
+++ b/tools/test-pkg
@@ -0,0 +1,197 @@
+#!/usr/bin/env bash
+set -e
+
+TOOLCHAINS_URL='http://autobuild.buildroot.org/toolchains/configs/toolchain-configs.csv'
+
+main() {
+    local o O opts
+    local cfg dir pkg random toolchain
+    local ret nb nb_skip nb_fail nb_legal nb_tc build_dir
+    local -a toolchains
+
+    o='hc:d:p:r:'
+    O='help,config-snippet:build-dir:package:,random:'
+    opts="$(getopt -n "${my_name}" -o "${o}" -l "${O}" -- "${@}")"
+    eval set -- "${opts}"
+
+    random=0
+    while [ ${#} -gt 0 ]; do
+        case "${1}" in
+        (-h|--help)
+            help; exit 0
+            ;;
+        (-c|--config-snippet)
+            cfg="${2}"; shift 2
+            ;;
+        (-d|--build-dir)
+            dir="${2}"; shift 2
+            ;;
+        (-p|--package)
+            pkg="${2}"; shift 2
+            ;;
+        (-r|--random)
+            random="${2}"; shift 2
+            ;;
+        (--)
+            shift; break
+            ;;
+        esac
+    done
+    if [ -z "${cfg}" ]; then
+        printf "error: no config snippet specified\n" >&2; exit 1
+    fi
+    if [ ! -e "${cfg}" ]; then
+        printf "error: %s: no such file\n" "${cfg}" >&2; exit 1
+    fi
+    if [ -z "${dir}" ]; then
+        dir="${HOME}/br-test-pkg"
+    fi
+
+    # Extract the URLs of the toolchains; drop internal toolchains
+    # E.g.: http://server/path/to/name.config,arch,libc
+    #  -->  http://server/path/to/name.config
+    toolchains=($(curl -s "${TOOLCHAINS_URL}" \
+                  |sed -r -e 's/,.*//; /internal/d;' \
+                  |if [ ${random} -gt 0 ]; then \
+                      sort -R |head -n ${random}
+                   else
+                      cat
+                   fi |sort
+                 )
+               )
+
+    nb_tc="${#toolchains[@]}"
+    if [ ${nb_tc} -eq 0 ]; then
+        printf "error: no toolchain found (networking issue?)\n" >&2; exit 1
+    fi
+
+    nb=0
+    nb_skip=0
+    nb_fail=0
+    nb_legal=0
+    for toolchainconfig in "${toolchains[@]}"; do
+        : $((nb++))
+        # Using basename(1) on a URL works nicely
+        toolchain="$(basename "${toolchainconfig}" .config)"
+        build_dir="${dir}/${toolchain}"
+        printf "%40s [%*d/%d]: " "${toolchain}" ${#nb_tc} ${nb} ${nb_tc}
+        build_one "${build_dir}" "${toolchainconfig}" "${cfg}" "${pkg}" && ret=0 || ret=${?}
+        case ${ret} in
+        (0) printf "OK\n";;
+        (1) : $((nb_skip++)); printf "SKIPPED\n";;
+        (2) : $((nb_fail++)); printf "FAILED\n";;
+        (3) : $((nb_legal++)); printf "FAILED\n";;
+        esac
+    done
+
+    printf "%d builds, %d skipped, %d build failed, %d legal-info failed\n" \
+        ${nb} ${nb_skip} ${nb_fail} ${nb_legal}
+}
+
+build_one() {
+    local dir="${1}"
+    local url="${2}"
+    local cfg="${3}"
+    local pkg="${4}"
+
+    mkdir -p "${dir}"
+
+    if ! curl -s "${url}" >"${dir}/.config"; then
+        return 2
+    fi
+
+    cat >>"${dir}/.config" <<-_EOF_
+ BR2_INIT_NONE=y
+ BR2_SYSTEM_BIN_SH_NONE=y
+ # BR2_PACKAGE_BUSYBOX is not set
+ # BR2_TARGET_ROOTFS_TAR is not set
+ _EOF_
+    cat "${cfg}" >>"${dir}/.config"
+
+    if ! make O="${dir}" olddefconfig > "${dir}/logfile" 2>&1; then
+        return 2
+    fi
+    # We want all the options from the snippet to be present as-is (set
+    # or not set) in the actual .config; if one of them is not, it means
+    # some dependency from the toolchain or arch is not available, in
+    # which case this config is untestable and we skip it.
+    # We don't care about the locale to sort in, as long as both sort are
+    # done in the same locale.
+    comm -23 <(sort "${cfg}") <(sort "${dir}/.config") >"${dir}/missing.config"
+    if [ -s "${dir}/missing.config" ]; then
+        return 1
+    fi
+    # Remove file, it's empty anyway.
+    rm -f "${dir}/missing.config"
+
+    if [ -n "${pkg}" ]; then
+        if ! make O="${dir}" "${pkg}-dirclean" >> "${dir}/logfile" 2>&1; then
+            return 2
+        fi
+    fi
+
+    # shellcheck disable=SC2086
+    if ! make O="${dir}" ${pkg} >> "${dir}/logfile" 2>&1; then
+        return 2
+    fi
+
+    # legal-info done systematically, because some packages have different
+    # sources depending on the configuration (e.g. lua-5.2 vs. lua-5.3)
+    if [ -n "${pkg}" ]; then
+        if ! make O="${dir}" "${pkg}-legal-info" >> "${dir}/logfile" 2>&1; then
+            return 3
+        fi
+    fi
+}
+
+help() {
+    cat <<_EOF_
+test-pkg: test-build a package against various toolchains and architectures
+
+The supplied config snippet is appended to each toolchain config, the
+resulting configuration is checked to ensure it still contains all options
+specified in the snippet; if any is missing, the build is skipped, on the
+assumption that the package under test requires a toolchain or architecture
+feature that is missing.
+
+In case failures are noticed, you can fix the package and just re-run the
+same command again; it will re-run the test where it failed. If you did
+specify a package (with -p), the package build dir will be removed first.
+
+The list of toolchains is retrieved from the Buildroot autobuilders, available
+at ${TOOLCHAINS_URL}.
+
+Options:
+
+    -h, --help
+        Print this help.
+
+    -c CFG, --config-snippet CFG
+        Use the CFG file as the source for the config snippet. This file
+        should contain all the config options required to build a package.
+
+    -d DIR, --build-dir DIR
+        Do the builds in directory DIR, one sub-dir per toolchain.
+
+    -p PKG, --package PKG
+        Test-build the package PKG, by running 'make PKG'; if not specified,
+        just runs 'make'.
+
+    -r N, --random N
+        Limit the tests to the N randomly selected toolchains, instead of
+        building with all toolchains.
+
+Example:
+
+    Testing libcec would require a config snippet that contains:
+        BR2_PACKAGE_LIBCEC=y
+
+    Testing libcurl with openSSL support would require a snippet such as:
+        BR2_PACKAGE_OPENSSL=y
+        BR2_PACKAGE_LIBCURL=y
+
+_EOF_
+}
+
+my_name="${0##*/}"
+main "${@}"
--
2.11.0

_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH 3/4 v2] tools: move get-developers out of support/scripts/

Yann E. MORIN-2
In reply to this post by Yann E. MORIN-2
Move it to the top-level tools/ directory, so that it is easier to
find for users.

Add a legacy symlink for those users who already used them, so as
not to break their habits.

Signed-off-by: "Yann E. MORIN" <[hidden email]>
Cc: Thomas Petazzoni <[hidden email]>
Cc: Arnout Vandecappelle <[hidden email]>
---
 docs/manual/contribute.txt                    |  2 +-
 support/scripts/get-developers                | 84 +--------------------------
 tools/00-README.txt                           |  5 ++
 tools/get-developers                          | 83 ++++++++++++++++++++++++++
 {support/scripts => tools}/getdeveloperlib.py |  0
 5 files changed, 90 insertions(+), 84 deletions(-)
 mode change 100755 => 120000 support/scripts/get-developers
 create mode 100755 tools/get-developers
 rename {support/scripts => tools}/getdeveloperlib.py (100%)

diff --git a/docs/manual/contribute.txt b/docs/manual/contribute.txt
index 719032bcc4..c10587726c 100644
--- a/docs/manual/contribute.txt
+++ b/docs/manual/contribute.txt
@@ -295,7 +295,7 @@ information). This tool reads your patches and outputs the appropriate
 +git send-email+ command to use:
 
 ---------------------
-$ ./support/scripts/get-developers outgoing/*
+$ ./tools/get-developers outgoing/*
 ---------------------
 
 Use the output of +get-developers+ to send your patches:
diff --git a/support/scripts/get-developers b/support/scripts/get-developers
deleted file mode 100755
index 40ed08ffe1..0000000000
--- a/support/scripts/get-developers
+++ /dev/null
@@ -1,83 +0,0 @@
-#!/usr/bin/env python
-
-import argparse
-import getdeveloperlib
-
-def parse_args():
-    parser = argparse.ArgumentParser()
-    parser.add_argument('patches', metavar='P', type=argparse.FileType('r'), nargs='*',
-                        help='list of patches (use - to read patches from stdin)')
-    parser.add_argument('-a', dest='architecture', action='store',
-                        help='find developers in charge of this architecture')
-    parser.add_argument('-p', dest='package', action='store',
-                        help='find developers in charge of this package')
-    parser.add_argument('-c', dest='check', action='store_const',
-                        const=True, help='list files not handled by any developer')
-    return parser.parse_args()
-
-def __main__():
-    devs = getdeveloperlib.parse_developers()
-    if devs is None:
-        sys.exit(1)
-    args = parse_args()
-
-    # Check that only one action is given
-    action = 0
-    if args.architecture is not None:
-        action += 1
-    if args.package is not None:
-        action += 1
-    if args.check:
-        action += 1
-    if len(args.patches) != 0:
-        action += 1
-    if action > 1:
-        print("Cannot do more than one action")
-        return
-    if action == 0:
-        print("No action specified")
-        return
-
-    # Handle the check action
-    if args.check:
-        files = getdeveloperlib.check_developers(devs)
-        for f in files:
-            print(f)
-
-    # Handle the architecture action
-    if args.architecture is not None:
-        for dev in devs:
-            if args.architecture in dev.architectures:
-                print(dev.name)
-        return
-
-    # Handle the package action
-    if args.package is not None:
-        for dev in devs:
-            if args.package in dev.packages:
-                print(dev.name)
-        return
-
-    # Handle the patches action
-    if len(args.patches) != 0:
-        (files, infras) = getdeveloperlib.analyze_patches(args.patches)
-        matching_devs = set()
-        for dev in devs:
-            # See if we have developers matching by package name
-            for f in files:
-                if dev.hasfile(f):
-                    matching_devs.add(dev.name)
-            # See if we have developers matching by package infra
-            for i in infras:
-                if i in dev.infras:
-                    matching_devs.add(dev.name)
-
-        result = "--to [hidden email]"
-        for dev in matching_devs:
-            result += " --cc \"%s\"" % dev
-
-        if result != "":
-            print("git send-email %s" % result)
-
-__main__()
-
diff --git a/support/scripts/get-developers b/support/scripts/get-developers
new file mode 120000
index 0000000000..7209567e15
--- /dev/null
+++ b/support/scripts/get-developers
@@ -0,0 +1 @@
+../../tools/get-developers
\ No newline at end of file
diff --git a/tools/00-README.txt b/tools/00-README.txt
index 092e0eac8e..2740b97e5b 100644
--- a/tools/00-README.txt
+++ b/tools/00-README.txt
@@ -8,6 +8,11 @@ brmake
     ("'br.log' in the current directory), and just ouputs the Buildroot
     messages (those lines starting with >>>) on stdout.
 
+get-developpers
+    a script to return the list of people interested in a specific part
+    of Buildroot, so they can be Cc:ed on a mail. Accepts a patch as
+    input, a package name or and architecture name.
+
 test-pkg
     a script that tests a specific package against a set of various
     toolchains, with the goal to detect toolchain-related dependencies
diff --git a/tools/get-developers b/tools/get-developers
new file mode 100755
index 0000000000..40ed08ffe1
--- /dev/null
+++ b/tools/get-developers
@@ -0,0 +1,83 @@
+#!/usr/bin/env python
+
+import argparse
+import getdeveloperlib
+
+def parse_args():
+    parser = argparse.ArgumentParser()
+    parser.add_argument('patches', metavar='P', type=argparse.FileType('r'), nargs='*',
+                        help='list of patches (use - to read patches from stdin)')
+    parser.add_argument('-a', dest='architecture', action='store',
+                        help='find developers in charge of this architecture')
+    parser.add_argument('-p', dest='package', action='store',
+                        help='find developers in charge of this package')
+    parser.add_argument('-c', dest='check', action='store_const',
+                        const=True, help='list files not handled by any developer')
+    return parser.parse_args()
+
+def __main__():
+    devs = getdeveloperlib.parse_developers()
+    if devs is None:
+        sys.exit(1)
+    args = parse_args()
+
+    # Check that only one action is given
+    action = 0
+    if args.architecture is not None:
+        action += 1
+    if args.package is not None:
+        action += 1
+    if args.check:
+        action += 1
+    if len(args.patches) != 0:
+        action += 1
+    if action > 1:
+        print("Cannot do more than one action")
+        return
+    if action == 0:
+        print("No action specified")
+        return
+
+    # Handle the check action
+    if args.check:
+        files = getdeveloperlib.check_developers(devs)
+        for f in files:
+            print(f)
+
+    # Handle the architecture action
+    if args.architecture is not None:
+        for dev in devs:
+            if args.architecture in dev.architectures:
+                print(dev.name)
+        return
+
+    # Handle the package action
+    if args.package is not None:
+        for dev in devs:
+            if args.package in dev.packages:
+                print(dev.name)
+        return
+
+    # Handle the patches action
+    if len(args.patches) != 0:
+        (files, infras) = getdeveloperlib.analyze_patches(args.patches)
+        matching_devs = set()
+        for dev in devs:
+            # See if we have developers matching by package name
+            for f in files:
+                if dev.hasfile(f):
+                    matching_devs.add(dev.name)
+            # See if we have developers matching by package infra
+            for i in infras:
+                if i in dev.infras:
+                    matching_devs.add(dev.name)
+
+        result = "--to [hidden email]"
+        for dev in matching_devs:
+            result += " --cc \"%s\"" % dev
+
+        if result != "":
+            print("git send-email %s" % result)
+
+__main__()
+
diff --git a/support/scripts/getdeveloperlib.py b/tools/getdeveloperlib.py
similarity index 100%
rename from support/scripts/getdeveloperlib.py
rename to tools/getdeveloperlib.py
--
2.11.0

_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH 4/4 v2] tools: move check-package out of support/scripts/

Yann E. MORIN-2
In reply to this post by Yann E. MORIN-2
Move it to the top-level tools/ directory, so that it is easier to
find for users.

Add a legacy symlink for those users who already used them, so as
not to break their habits.

Signed-off-by: "Yann E. MORIN" <[hidden email]>
Cc: Thomas Petazzoni <[hidden email]>
Cc: Arnout Vandecappelle <[hidden email]>
---
 support/scripts/check-package                      | 145 +--------------------
 support/scripts/pkg-stats                          |   2 +-
 tools/00-README.txt                                |   4 +
 tools/check-package                                | 144 ++++++++++++++++++++
 .../scripts => tools}/checkpackagelib/__init__.py  |   0
 {support/scripts => tools}/checkpackagelib/base.py |   0
 {support/scripts => tools}/checkpackagelib/lib.py  |   0
 .../checkpackagelib/lib_config.py                  |   0
 .../scripts => tools}/checkpackagelib/lib_hash.py  |   0
 .../scripts => tools}/checkpackagelib/lib_mk.py    |   0
 .../scripts => tools}/checkpackagelib/lib_patch.py |   0
 .../scripts => tools}/checkpackagelib/readme.txt   |  10 +-
 12 files changed, 155 insertions(+), 150 deletions(-)
 mode change 100755 => 120000 support/scripts/check-package
 create mode 100755 tools/check-package
 rename {support/scripts => tools}/checkpackagelib/__init__.py (100%)
 rename {support/scripts => tools}/checkpackagelib/base.py (100%)
 rename {support/scripts => tools}/checkpackagelib/lib.py (100%)
 rename {support/scripts => tools}/checkpackagelib/lib_config.py (100%)
 rename {support/scripts => tools}/checkpackagelib/lib_hash.py (100%)
 rename {support/scripts => tools}/checkpackagelib/lib_mk.py (100%)
 rename {support/scripts => tools}/checkpackagelib/lib_patch.py (100%)
 rename {support/scripts => tools}/checkpackagelib/readme.txt (91%)

diff --git a/support/scripts/check-package b/support/scripts/check-package
deleted file mode 100755
index 74ea46f819..0000000000
--- a/support/scripts/check-package
+++ /dev/null
@@ -1,144 +0,0 @@
-#!/usr/bin/env python
-# See support/scripts/checkpackagelib/readme.txt before editing this file.
-
-from __future__ import print_function
-import argparse
-import inspect
-import re
-import sys
-
-import checkpackagelib.lib_config
-import checkpackagelib.lib_hash
-import checkpackagelib.lib_mk
-import checkpackagelib.lib_patch
-
-VERBOSE_LEVEL_TO_SHOW_IGNORED_FILES = 3
-flags = None  # Command line arguments.
-
-
-def parse_args():
-    parser = argparse.ArgumentParser()
-
-    # Do not use argparse.FileType("r") here because only files with known
-    # format will be open based on the filename.
-    parser.add_argument("files", metavar="F", type=str, nargs="*",
-                        help="list of files")
-
-    parser.add_argument("--manual-url", action="store",
-                        default="http://nightly.buildroot.org/",
-                        help="default: %(default)s")
-    parser.add_argument("--verbose", "-v", action="count", default=0)
-
-    # Now the debug options in the order they are processed.
-    parser.add_argument("--include-only", dest="include_list", action="append",
-                        help="run only the specified functions (debug)")
-    parser.add_argument("--exclude", dest="exclude_list", action="append",
-                        help="do not run the specified functions (debug)")
-    parser.add_argument("--dry-run", action="store_true", help="print the "
-                        "functions that would be called for each file (debug)")
-
-    return parser.parse_args()
-
-
-CONFIG_IN_FILENAME = re.compile("/Config\.\S*$")
-FILE_IS_FROM_A_PACKAGE = re.compile("package/[^/]*/")
-
-
-def get_lib_from_filename(fname):
-    if FILE_IS_FROM_A_PACKAGE.search(fname) is None:
-        return None
-    if CONFIG_IN_FILENAME.search(fname):
-        return checkpackagelib.lib_config
-    if fname.endswith(".hash"):
-        return checkpackagelib.lib_hash
-    if fname.endswith(".mk"):
-        return checkpackagelib.lib_mk
-    if fname.endswith(".patch"):
-        return checkpackagelib.lib_patch
-    return None
-
-
-def is_a_check_function(m):
-    if not inspect.isclass(m):
-        return False
-    # do not call the base class
-    if m.__name__.startswith("_"):
-        return False
-    if flags.include_list and m.__name__ not in flags.include_list:
-        return False
-    if flags.exclude_list and m.__name__ in flags.exclude_list:
-        return False
-    return True
-
-
-def print_warnings(warnings):
-    # Avoid the need to use 'return []' at the end of every check function.
-    if warnings is None:
-        return 0  # No warning generated.
-
-    for level, message in enumerate(warnings):
-        if flags.verbose >= level:
-            print(message.replace("\t", "< tab  >").rstrip())
-    return 1  # One more warning to count.
-
-
-def check_file_using_lib(fname):
-    # Count number of warnings generated and lines processed.
-    nwarnings = 0
-    nlines = 0
-
-    lib = get_lib_from_filename(fname)
-    if not lib:
-        if flags.verbose >= VERBOSE_LEVEL_TO_SHOW_IGNORED_FILES:
-            print("{}: ignored".format(fname))
-        return nwarnings, nlines
-    classes = inspect.getmembers(lib, is_a_check_function)
-
-    if flags.dry_run:
-        functions_to_run = [c[0] for c in classes]
-        print("{}: would run: {}".format(fname, functions_to_run))
-        return nwarnings, nlines
-
-    objects = [c[1](fname, flags.manual_url) for c in classes]
-
-    for cf in objects:
-        nwarnings += print_warnings(cf.before())
-    for lineno, text in enumerate(open(fname, "r").readlines()):
-        nlines += 1
-        for cf in objects:
-            nwarnings += print_warnings(cf.check_line(lineno + 1, text))
-    for cf in objects:
-        nwarnings += print_warnings(cf.after())
-
-    return nwarnings, nlines
-
-
-def __main__():
-    global flags
-    flags = parse_args()
-
-    if len(flags.files) == 0:
-        print("No files to check style")
-        sys.exit(1)
-
-    # Accumulate number of warnings generated and lines processed.
-    total_warnings = 0
-    total_lines = 0
-
-    for fname in flags.files:
-        nwarnings, nlines = check_file_using_lib(fname)
-        total_warnings += nwarnings
-        total_lines += nlines
-
-    # The warning messages are printed to stdout and can be post-processed
-    # (e.g. counted by 'wc'), so for stats use stderr. Wait all warnings are
-    # printed, for the case there are many of them, before printing stats.
-    sys.stdout.flush()
-    print("{} lines processed".format(total_lines), file=sys.stderr)
-    print("{} warnings generated".format(total_warnings), file=sys.stderr)
-
-    if total_warnings > 0:
-        sys.exit(1)
-
-
-__main__()
diff --git a/support/scripts/check-package b/support/scripts/check-package
new file mode 120000
index 0000000000..16a6051e3a
--- /dev/null
+++ b/support/scripts/check-package
@@ -0,0 +1 @@
+../../tools/check-package
\ No newline at end of file
diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats
index 4cf1f82518..f827877052 100755
--- a/support/scripts/pkg-stats
+++ b/support/scripts/pkg-stats
@@ -337,7 +337,7 @@ for i in $(find boot/ linux/ package/ toolchain/ -name '*.mk' | sort) ; do
     fi
 
     file_list=$(find ${package_dir} -name '*.mk' -o -name '*.in*' -o -name '*.hash')
-    nwarnings=$(./support/scripts/check-package ${file_list} 2>&1 | sed '/\([0-9]*\) warnings generated/!d; s//\1/')
+    nwarnings=$(./tools/check-package ${file_list} 2>&1 | sed '/\([0-9]*\) warnings generated/!d; s//\1/')
     if [ ${nwarnings} -eq 0 ] ; then
  echo "<td class=\"centered correct\">${nwarnings}</td>"
     else
diff --git a/tools/00-README.txt b/tools/00-README.txt
index 2740b97e5b..5e563cc59a 100644
--- a/tools/00-README.txt
+++ b/tools/00-README.txt
@@ -8,6 +8,10 @@ brmake
     ("'br.log' in the current directory), and just ouputs the Buildroot
     messages (those lines starting with >>>) on stdout.
 
+check-package
+    a script that checks the coding style of a package's Config.in and
+    .mk files, and also tests them for various types of typoes.
+
 get-developpers
     a script to return the list of people interested in a specific part
     of Buildroot, so they can be Cc:ed on a mail. Accepts a patch as
diff --git a/tools/check-package b/tools/check-package
new file mode 100755
index 0000000000..cdbac94929
--- /dev/null
+++ b/tools/check-package
@@ -0,0 +1,144 @@
+#!/usr/bin/env python
+# See tools/checkpackagelib/readme.txt before editing this file.
+
+from __future__ import print_function
+import argparse
+import inspect
+import re
+import sys
+
+import checkpackagelib.lib_config
+import checkpackagelib.lib_hash
+import checkpackagelib.lib_mk
+import checkpackagelib.lib_patch
+
+VERBOSE_LEVEL_TO_SHOW_IGNORED_FILES = 3
+flags = None  # Command line arguments.
+
+
+def parse_args():
+    parser = argparse.ArgumentParser()
+
+    # Do not use argparse.FileType("r") here because only files with known
+    # format will be open based on the filename.
+    parser.add_argument("files", metavar="F", type=str, nargs="*",
+                        help="list of files")
+
+    parser.add_argument("--manual-url", action="store",
+                        default="http://nightly.buildroot.org/",
+                        help="default: %(default)s")
+    parser.add_argument("--verbose", "-v", action="count", default=0)
+
+    # Now the debug options in the order they are processed.
+    parser.add_argument("--include-only", dest="include_list", action="append",
+                        help="run only the specified functions (debug)")
+    parser.add_argument("--exclude", dest="exclude_list", action="append",
+                        help="do not run the specified functions (debug)")
+    parser.add_argument("--dry-run", action="store_true", help="print the "
+                        "functions that would be called for each file (debug)")
+
+    return parser.parse_args()
+
+
+CONFIG_IN_FILENAME = re.compile("/Config\.\S*$")
+FILE_IS_FROM_A_PACKAGE = re.compile("package/[^/]*/")
+
+
+def get_lib_from_filename(fname):
+    if FILE_IS_FROM_A_PACKAGE.search(fname) is None:
+        return None
+    if CONFIG_IN_FILENAME.search(fname):
+        return checkpackagelib.lib_config
+    if fname.endswith(".hash"):
+        return checkpackagelib.lib_hash
+    if fname.endswith(".mk"):
+        return checkpackagelib.lib_mk
+    if fname.endswith(".patch"):
+        return checkpackagelib.lib_patch
+    return None
+
+
+def is_a_check_function(m):
+    if not inspect.isclass(m):
+        return False
+    # do not call the base class
+    if m.__name__.startswith("_"):
+        return False
+    if flags.include_list and m.__name__ not in flags.include_list:
+        return False
+    if flags.exclude_list and m.__name__ in flags.exclude_list:
+        return False
+    return True
+
+
+def print_warnings(warnings):
+    # Avoid the need to use 'return []' at the end of every check function.
+    if warnings is None:
+        return 0  # No warning generated.
+
+    for level, message in enumerate(warnings):
+        if flags.verbose >= level:
+            print(message.replace("\t", "< tab  >").rstrip())
+    return 1  # One more warning to count.
+
+
+def check_file_using_lib(fname):
+    # Count number of warnings generated and lines processed.
+    nwarnings = 0
+    nlines = 0
+
+    lib = get_lib_from_filename(fname)
+    if not lib:
+        if flags.verbose >= VERBOSE_LEVEL_TO_SHOW_IGNORED_FILES:
+            print("{}: ignored".format(fname))
+        return nwarnings, nlines
+    classes = inspect.getmembers(lib, is_a_check_function)
+
+    if flags.dry_run:
+        functions_to_run = [c[0] for c in classes]
+        print("{}: would run: {}".format(fname, functions_to_run))
+        return nwarnings, nlines
+
+    objects = [c[1](fname, flags.manual_url) for c in classes]
+
+    for cf in objects:
+        nwarnings += print_warnings(cf.before())
+    for lineno, text in enumerate(open(fname, "r").readlines()):
+        nlines += 1
+        for cf in objects:
+            nwarnings += print_warnings(cf.check_line(lineno + 1, text))
+    for cf in objects:
+        nwarnings += print_warnings(cf.after())
+
+    return nwarnings, nlines
+
+
+def __main__():
+    global flags
+    flags = parse_args()
+
+    if len(flags.files) == 0:
+        print("No files to check style")
+        sys.exit(1)
+
+    # Accumulate number of warnings generated and lines processed.
+    total_warnings = 0
+    total_lines = 0
+
+    for fname in flags.files:
+        nwarnings, nlines = check_file_using_lib(fname)
+        total_warnings += nwarnings
+        total_lines += nlines
+
+    # The warning messages are printed to stdout and can be post-processed
+    # (e.g. counted by 'wc'), so for stats use stderr. Wait all warnings are
+    # printed, for the case there are many of them, before printing stats.
+    sys.stdout.flush()
+    print("{} lines processed".format(total_lines), file=sys.stderr)
+    print("{} warnings generated".format(total_warnings), file=sys.stderr)
+
+    if total_warnings > 0:
+        sys.exit(1)
+
+
+__main__()
diff --git a/support/scripts/checkpackagelib/__init__.py b/tools/checkpackagelib/__init__.py
similarity index 100%
rename from support/scripts/checkpackagelib/__init__.py
rename to tools/checkpackagelib/__init__.py
diff --git a/support/scripts/checkpackagelib/base.py b/tools/checkpackagelib/base.py
similarity index 100%
rename from support/scripts/checkpackagelib/base.py
rename to tools/checkpackagelib/base.py
diff --git a/support/scripts/checkpackagelib/lib.py b/tools/checkpackagelib/lib.py
similarity index 100%
rename from support/scripts/checkpackagelib/lib.py
rename to tools/checkpackagelib/lib.py
diff --git a/support/scripts/checkpackagelib/lib_config.py b/tools/checkpackagelib/lib_config.py
similarity index 100%
rename from support/scripts/checkpackagelib/lib_config.py
rename to tools/checkpackagelib/lib_config.py
diff --git a/support/scripts/checkpackagelib/lib_hash.py b/tools/checkpackagelib/lib_hash.py
similarity index 100%
rename from support/scripts/checkpackagelib/lib_hash.py
rename to tools/checkpackagelib/lib_hash.py
diff --git a/support/scripts/checkpackagelib/lib_mk.py b/tools/checkpackagelib/lib_mk.py
similarity index 100%
rename from support/scripts/checkpackagelib/lib_mk.py
rename to tools/checkpackagelib/lib_mk.py
diff --git a/support/scripts/checkpackagelib/lib_patch.py b/tools/checkpackagelib/lib_patch.py
similarity index 100%
rename from support/scripts/checkpackagelib/lib_patch.py
rename to tools/checkpackagelib/lib_patch.py
diff --git a/support/scripts/checkpackagelib/readme.txt b/tools/checkpackagelib/readme.txt
similarity index 91%
rename from support/scripts/checkpackagelib/readme.txt
rename to tools/checkpackagelib/readme.txt
index 0444d17992..8012c72e39 100644
--- a/support/scripts/checkpackagelib/readme.txt
+++ b/tools/checkpackagelib/readme.txt
@@ -57,19 +57,19 @@ Some hints when changing this code:
 Usage examples:
 - to get a list of check functions that would be called without actually
   calling them you can use the --dry-run option:
-$ support/scripts/check-package --dry-run package/yourfavorite/*
+$ tools/check-package --dry-run package/yourfavorite/*
 
 - when you just added a new check function, e.g. Something, check how it behaves
   for all current packages:
-$ support/scripts/check-package --include-only Something $(find package -type f)
+$ tools/check-package --include-only Something $(find package -type f)
 
 - the effective processing time (when the .pyc were already generated and all
   files to be processed are cached in the RAM) should stay in the order of few
   seconds:
-$ support/scripts/check-package $(find package -type f) >/dev/null ; \
-  time support/scripts/check-package $(find package -type f) >/dev/null
+$ tools/check-package $(find package -type f) >/dev/null ; \
+  time tools/check-package $(find package -type f) >/dev/null
 
 - vim users can navigate the warnings (most editors probably have similar
   function) since warnings are generated in the form 'path/file:line: warning':
 $ find package/ -name 'Config.*' > filelist && vim -c \
-  'set makeprg=support/scripts/check-package\ $(cat\ filelist)' -c make -c copen
+  'set makeprg=tools/check-package\ $(cat\ filelist)' -c make -c copen
--
2.11.0

_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH 2/4 v2] tools: move test-pkg out of support/scripts

Thomas Petazzoni-2
In reply to this post by Yann E. MORIN-2
Hello,

On Sun, 18 Jun 2017 16:01:43 +0200, Yann E. MORIN wrote:

>  docs/manual/adding-packages-tips.txt |  12 +--
>  support/scripts/test-pkg             | 198 +----------------------------------
>  tools/00-README.txt                  |   5 +
>  tools/test-pkg                       | 197 ++++++++++++++++++++++++++++++++++
>  4 files changed, 209 insertions(+), 203 deletions(-)
>  mode change 100755 => 120000 support/scripts/test-pkg
>  create mode 100755 tools/test-pkg

No strong opinion on whether such a move is desirable, but at the very
list I believe we should not add this 00-README.txt file. Nowhere else
in the tree we have files named like this, and I don't think we should
start adding these.

Best regards,

Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH 0/4 v2] tools: add a directory to contain usefull user-facing tools

Thomas Petazzoni-2
In reply to this post by Yann E. MORIN-2
Hello,

On Sun, 18 Jun 2017 16:01:40 +0200, Yann E. MORIN wrote:

> Currently, we provide a few user-facing utilities, like get-developers,
> in support/scripts/ . But this directory also contains internal scripts
> that a user should not be directly concerned with. Besides, it is
> two-level deep in the hierarchy, which is not really nice.
>
> So, we introduce tools/ as a top-level directory, with the goal to store
> and expose all user-facing utilities, while keeping our internal scripts
> in support/scripts/ .
>
> The first patch in the series introduces a new utility, brmake, a
> wrapper around make that stores all the output to a log file and
> displays only the >>> lines.
>
> The second and subsequent patches each move a few user-facing utilities
> out of support/scripts/ and into tools/ .

I am wondering if there is a very clear cut boundary between
user-facing tools and non-user facing tools.

For example, graph-build-time or graph-depends are internally used by
the Buildroot Makefile, but may also be used directly. The size-stats
tool is also used internally by the Buildroot Makefile, but
size-stats-compare is meant to be used directly.

What about support/scripts/genimage.sh ?

Best regards,

Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH 2/4 v2] tools: move test-pkg out of support/scripts

Yann E. MORIN-2
In reply to this post by Thomas Petazzoni-2
Thomas, All,

On 2017-06-18 16:12 +0200, Thomas Petazzoni spake thusly:

> On Sun, 18 Jun 2017 16:01:43 +0200, Yann E. MORIN wrote:
> >  docs/manual/adding-packages-tips.txt |  12 +--
> >  support/scripts/test-pkg             | 198 +----------------------------------
> >  tools/00-README.txt                  |   5 +
> >  tools/test-pkg                       | 197 ++++++++++++++++++++++++++++++++++
> >  4 files changed, 209 insertions(+), 203 deletions(-)
> >  mode change 100755 => 120000 support/scripts/test-pkg
> >  create mode 100755 tools/test-pkg
>
> No strong opinion on whether such a move is desirable,

That was a proposal by Arnout:
    http://lists.busybox.net/pipermail/buildroot/2016-October/174290.html

> but at the very
> list I believe we should not add this 00-README.txt file. Nowhere else
> in the tree we have files named like this, and I don't think we should
> start adding these.

I'm OK for dropping it.

Regards,
Yann E. MORIN.

--
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH 0/4 v2] tools: add a directory to contain usefull user-facing tools

Yann E. MORIN-2
In reply to this post by Thomas Petazzoni-2
Thomas, All,

On 2017-06-18 16:15 +0200, Thomas Petazzoni spake thusly:

> On Sun, 18 Jun 2017 16:01:40 +0200, Yann E. MORIN wrote:
> > Currently, we provide a few user-facing utilities, like get-developers,
> > in support/scripts/ . But this directory also contains internal scripts
> > that a user should not be directly concerned with. Besides, it is
> > two-level deep in the hierarchy, which is not really nice.
> >
> > So, we introduce tools/ as a top-level directory, with the goal to store
> > and expose all user-facing utilities, while keeping our internal scripts
> > in support/scripts/ .
> >
> > The first patch in the series introduces a new utility, brmake, a
> > wrapper around make that stores all the output to a log file and
> > displays only the >>> lines.
> >
> > The second and subsequent patches each move a few user-facing utilities
> > out of support/scripts/ and into tools/ .
>
> I am wondering if there is a very clear cut boundary between
> user-facing tools and non-user facing tools.

Yes, the frontier is not trivial.

I'd say that if a tool is meant to be called by the user, then it
belongs to tools/. If it is not meant to be alled by a user (even if it
could be so) then it belongs to support/scripts/.

> For example, graph-build-time or graph-depends are internally used by
> the Buildroot Makefile, but may also be used directly.

The primary use for those scripts is still an internal use. They are
supposed to be called from our infra.

> The size-stats
> tool is also used internally by the Buildroot Makefile, but
> size-stats-compare is meant to be used directly.

Then the former stays in support/scripts/ while the latter should move
to tools/.

> What about support/scripts/genimage.sh ?

I would leave it to support/scripts/. It is an internal helper./
Besides, it needs a few of our variables (TARGET_DIR and BINARIES_DIR),
so I see it as internal.

Of course, I might not have moved all that can/should be moved. This
series serves as a base for further discussion.

See the initial submission:

    [PATCH] contrib: add a directory to store "contrib" stuff
    http://lists.busybox.net/pipermail/buildroot/2016-October/174238.html

Regards,
Yann E. MORIN.

> Best regards,
>
> Thomas
> --
> Thomas Petazzoni, CTO, Free Electrons
> Embedded Linux and Kernel engineering
> http://free-electrons.com

--
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH 1/4 v2] tools: add a directory to store some useful user-facing tools

Ricardo Martincoski
In reply to this post by Yann E. MORIN-2
Hello,

On Sun, Jun 18, 2017 at 11:01 AM, Yann E. MORIN wrote:

> Add a tools directory to store useful little scripts, tools, etc...
> Users may add that directory in their PATH if they want to, but this
> is not mandatory.
>
> Add the first tool: brmake, a small script that redirects the build
> output log to a file, keeping just Buildroot's own messages, with the
> date+time added at the start of the line.
>
> We need to unbuffer the output of make so that, when the script is
> interrupted (SIGINT, ^C), there is no lingering output not yet digested
> by the logger loop.
>
> Signed-off-by: "Yann E. MORIN" <[hidden email]>
> Cc: Maxime Hadjinlian <[hidden email]>
> Cc: Arnout Vandecappelle <[hidden email]>
I tested the script and it works very well in general.

- I successfully built the qemu_arm_versatile_defconfig using the script,
pressing CTRL+C a few times and resuming by calling it again.
- Also, by inserting an error to a package I got the expected behaviour: the
error message is in the log file and the script returned the error code.
- Tested using 'GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-gnu)'

BUT if it is called without a .config file, it hangs forever waiting for input,
and the menuconfig is sent to the log file.
Maybe we can bailout if no .config is present.

Also, 2 small nits below.

> +++ b/tools/00-README.txt
> @@ -0,0 +1,9 @@
> +This directory contains various useful scripts and tools for working
> +with Buildroot. You need not add this directory in your PATH to use
> +any of those tools, but you may do so if you want.
> +
> +brmake
> +    a script that can be run instead of make, that prepends the date in
> +    front of each line, redirects all of the build output to a file
> +    ("'br.log' in the current directory), and just ouputs the Buildroot
Maybe this file will be dropped as discussed in
http://patchwork.ozlabs.org/patch/777446/
If not, there is a typo

ouputs -> outputs

> +    messages (those lines starting with >>>) on stdout.
> diff --git a/tools/brmake b/tools/brmake
> new file mode 100755
> index 0000000000..9f7b4bc65f
> --- /dev/null
> +++ b/tools/brmake
> @@ -0,0 +1,37 @@
> +#!/bin/bash
> +# (C) 2016, "Yann E. MORIN" <[hidden email]>
> +# License: WTFPL, https://spdx.org/licenses/WTFPL.html
> +
> +main() {
> +    local ret start d h m mf fs
fs is unused.

Regards,
Ricardo
_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH 4/4 v2] tools: move check-package out of support/scripts/

Ricardo Martincoski
In reply to this post by Yann E. MORIN-2
Hello,

On Sun, Jun 18, 2017 at 11:01 AM, Yann E. MORIN wrote:

[snip]
>  rename {support/scripts => tools}/checkpackagelib/base.py (100%)
>  rename {support/scripts => tools}/checkpackagelib/lib.py (100%)
>  rename {support/scripts => tools}/checkpackagelib/lib_config.py (100%)
>  rename {support/scripts => tools}/checkpackagelib/lib_hash.py (100%)
>  rename {support/scripts => tools}/checkpackagelib/lib_mk.py (100%)
>  rename {support/scripts => tools}/checkpackagelib/lib_patch.py (100%)
>  rename {support/scripts => tools}/checkpackagelib/readme.txt (91%)

nit: there are 6 more comments using the old path.

-# See support/scripts/checkpackagelib/readme.txt before editing this file.
+# See tools/checkpackagelib/readme.txt before editing this file.

It could be fixed when applying using this sed:
sed -e 's,support/scripts/,tools/,g' -i tools/checkpackagelib/*.py

Another option is to remove the 6 comments if you think they are too repetitive.

Regards,
Ricardo
_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH 4/4 v2] tools: move check-package out of support/scripts/

Ricardo Martincoski
In reply to this post by Yann E. MORIN-2
Hello,

On Sun, Jun 18, 2017 at 11:01 AM, Yann E. MORIN wrote:

> ---
>  support/scripts/check-package                      | 145 +--------------------
>  support/scripts/pkg-stats                          |   2 +-
>  tools/00-README.txt                                |   4 +
>  tools/check-package                                | 144 ++++++++++++++++++++
>  .../scripts => tools}/checkpackagelib/__init__.py  |   0
>  {support/scripts => tools}/checkpackagelib/base.py |   0
>  {support/scripts => tools}/checkpackagelib/lib.py  |   0
>  .../checkpackagelib/lib_config.py                  |   0
>  .../scripts => tools}/checkpackagelib/lib_hash.py  |   0
>  .../scripts => tools}/checkpackagelib/lib_mk.py    |   0
>  .../scripts => tools}/checkpackagelib/lib_patch.py |   0
>  .../scripts => tools}/checkpackagelib/readme.txt   |  10 +-
>  12 files changed, 155 insertions(+), 150 deletions(-)
>  mode change 100755 => 120000 support/scripts/check-package
>  create mode 100755 tools/check-package
>  rename {support/scripts => tools}/checkpackagelib/__init__.py (100%)
>  rename {support/scripts => tools}/checkpackagelib/base.py (100%)
>  rename {support/scripts => tools}/checkpackagelib/lib.py (100%)
>  rename {support/scripts => tools}/checkpackagelib/lib_config.py (100%)
>  rename {support/scripts => tools}/checkpackagelib/lib_hash.py (100%)
>  rename {support/scripts => tools}/checkpackagelib/lib_mk.py (100%)
>  rename {support/scripts => tools}/checkpackagelib/lib_patch.py (100%)
>  rename {support/scripts => tools}/checkpackagelib/readme.txt (91%)
I forgot earlier...

Also the entry in the DEVELOPERS file must be updated
F: support/scripts/check*package*

Probably if I didn't used '*' it would be easier to find.
I didn't tested but I assume we could use
F: tools/check-package
F: tools/checkpackagelib/

Regards,
Ricardo
_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH 1/4 v2] tools: add a directory to store some useful user-facing tools

Yann E. MORIN-2
In reply to this post by Ricardo Martincoski
Ricardo, All,

On 2017-06-18 22:11 -0300, Ricardo Martincoski spake thusly:

> On Sun, Jun 18, 2017 at 11:01 AM, Yann E. MORIN wrote:
> > Add a tools directory to store useful little scripts, tools, etc...
> > Users may add that directory in their PATH if they want to, but this
> > is not mandatory.
> >
> > Add the first tool: brmake, a small script that redirects the build
> > output log to a file, keeping just Buildroot's own messages, with the
> > date+time added at the start of the line.
> >
> > We need to unbuffer the output of make so that, when the script is
> > interrupted (SIGINT, ^C), there is no lingering output not yet digested
> > by the logger loop.
> >
> > Signed-off-by: "Yann E. MORIN" <[hidden email]>
> > Cc: Maxime Hadjinlian <[hidden email]>
> > Cc: Arnout Vandecappelle <[hidden email]>
>
> I tested the script and it works very well in general.
>
> - I successfully built the qemu_arm_versatile_defconfig using the script,
> pressing CTRL+C a few times and resuming by calling it again.
> - Also, by inserting an error to a package I got the expected behaviour: the
> error message is in the log file and the script returned the error code.
> - Tested using 'GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-gnu)'
>
> BUT if it is called without a .config file, it hangs forever waiting for input,
> and the menuconfig is sent to the log file.

Indeed, my use-case is to only use it in an already-configured tree.

> Maybe we can bailout if no .config is present.

Hmmm... It's not trivial do detect, because the .config file is not
necessarily in the current directory, for example:

    brmake -C /pat/to/buil-dir

and I don't want to write a make option interpeter either...

But I'll see what I can do.

> Also, 2 small nits below.
>
> > +++ b/tools/00-README.txt
> > @@ -0,0 +1,9 @@
> > +This directory contains various useful scripts and tools for working
> > +with Buildroot. You need not add this directory in your PATH to use
> > +any of those tools, but you may do so if you want.
> > +
> > +brmake
> > +    a script that can be run instead of make, that prepends the date in
> > +    front of each line, redirects all of the build output to a file
> > +    ("'br.log' in the current directory), and just ouputs the Buildroot
>
> Maybe this file will be dropped as discussed in
> http://patchwork.ozlabs.org/patch/777446/
> If not, there is a typo
>
> ouputs -> outputs

Yup, I already dropped the file locally.

> > +    messages (those lines starting with >>>) on stdout.
> > diff --git a/tools/brmake b/tools/brmake
> > new file mode 100755
> > index 0000000000..9f7b4bc65f
> > --- /dev/null
> > +++ b/tools/brmake
> > @@ -0,0 +1,37 @@
> > +#!/bin/bash
> > +# (C) 2016, "Yann E. MORIN" <[hidden email]>
> > +# License: WTFPL, https://spdx.org/licenses/WTFPL.html
> > +
> > +main() {
> > +    local ret start d h m mf fs
>
> fs is unused.

OK. Thanks for the review and tests! :-)

Regards,
Yann E. MORIN.

> Regards,
> Ricardo


--
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH 4/4 v2] tools: move check-package out of support/scripts/

Yann E. MORIN-2
In reply to this post by Ricardo Martincoski
Ricardo, All,

On 2017-06-18 22:13 -0300, Ricardo Martincoski spake thusly:

> On Sun, Jun 18, 2017 at 11:01 AM, Yann E. MORIN wrote:
> [snip]
> >  rename {support/scripts => tools}/checkpackagelib/base.py (100%)
> >  rename {support/scripts => tools}/checkpackagelib/lib.py (100%)
> >  rename {support/scripts => tools}/checkpackagelib/lib_config.py (100%)
> >  rename {support/scripts => tools}/checkpackagelib/lib_hash.py (100%)
> >  rename {support/scripts => tools}/checkpackagelib/lib_mk.py (100%)
> >  rename {support/scripts => tools}/checkpackagelib/lib_patch.py (100%)
> >  rename {support/scripts => tools}/checkpackagelib/readme.txt (91%)
>
> nit: there are 6 more comments using the old path.

Damn, right.

> -# See support/scripts/checkpackagelib/readme.txt before editing this file.
> +# See tools/checkpackagelib/readme.txt before editing this file.
>
> It could be fixed when applying using this sed:
> sed -e 's,support/scripts/,tools/,g' -i tools/checkpackagelib/*.py
>
> Another option is to remove the 6 comments if you think they are too repetitive.

Fixed.

Thanks for the review! :-)

Regards,
Yann E. MORIN.

--
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH 4/4 v2] tools: move check-package out of support/scripts/

Yann E. MORIN-2
In reply to this post by Ricardo Martincoski
Ricardo, All,

On 2017-06-19 00:22 -0300, Ricardo Martincoski spake thusly:

> On Sun, Jun 18, 2017 at 11:01 AM, Yann E. MORIN wrote:
>
> > ---
> >  support/scripts/check-package                      | 145 +--------------------
> >  support/scripts/pkg-stats                          |   2 +-
> >  tools/00-README.txt                                |   4 +
> >  tools/check-package                                | 144 ++++++++++++++++++++
> >  .../scripts => tools}/checkpackagelib/__init__.py  |   0
> >  {support/scripts => tools}/checkpackagelib/base.py |   0
> >  {support/scripts => tools}/checkpackagelib/lib.py  |   0
> >  .../checkpackagelib/lib_config.py                  |   0
> >  .../scripts => tools}/checkpackagelib/lib_hash.py  |   0
> >  .../scripts => tools}/checkpackagelib/lib_mk.py    |   0
> >  .../scripts => tools}/checkpackagelib/lib_patch.py |   0
> >  .../scripts => tools}/checkpackagelib/readme.txt   |  10 +-
> >  12 files changed, 155 insertions(+), 150 deletions(-)
> >  mode change 100755 => 120000 support/scripts/check-package
> >  create mode 100755 tools/check-package
> >  rename {support/scripts => tools}/checkpackagelib/__init__.py (100%)
> >  rename {support/scripts => tools}/checkpackagelib/base.py (100%)
> >  rename {support/scripts => tools}/checkpackagelib/lib.py (100%)
> >  rename {support/scripts => tools}/checkpackagelib/lib_config.py (100%)
> >  rename {support/scripts => tools}/checkpackagelib/lib_hash.py (100%)
> >  rename {support/scripts => tools}/checkpackagelib/lib_mk.py (100%)
> >  rename {support/scripts => tools}/checkpackagelib/lib_patch.py (100%)
> >  rename {support/scripts => tools}/checkpackagelib/readme.txt (91%)
>
> I forgot earlier...
>
> Also the entry in the DEVELOPERS file must be updated
> F: support/scripts/check*package*
>
> Probably if I didn't used '*' it would be easier to find.
> I didn't tested but I assume we could use
> F: tools/check-package
> F: tools/checkpackagelib/

I'll fix as you suggest, to drop the midle wildcard.

Also, you happen to have two entries in the DEVELOPPERS file, with two
different emails:

  1368 N:» Ricardo Martincoski <[hidden email]>
  1369 F:» package/atop/

and

  1374 N:» Ricardo Martincoski <[hidden email]>
  1375 F:» support/scripts/check*package*

Is that expected?

Regards,
Yann E. MORIN.

--
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH 4/4 v2] tools: move check-package out of support/scripts/

Ricardo Martincoski-2
Hello,

>> On Sun, Jun 18, 2017 at 11:01 AM, Yann E. MORIN wrote:
> Also, you happen to have two entries in the DEVELOPPERS file, with two
> different emails:
>
>  1368 N:» Ricardo Martincoski <[hidden email]>
>  1369 F:» package/atop/
>
> and
>
>  1374 N:» Ricardo Martincoski <[hidden email]>
>  1375 F:» support/scripts/check*package*
>
> Is that expected?

Yes, professional and personal entries.

Regards,
Ricardo
_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH 1/4 v2] tools: add a directory to store some useful user-facing tools

Ricardo Martincoski
In reply to this post by Yann E. MORIN-2
Hello,

On Mon, Jun 19, 2017 at 12:42 PM, Yann E. MORIN wrote:

> On 2017-06-18 22:11 -0300, Ricardo Martincoski spake thusly:
[snip]

>> BUT if it is called without a .config file, it hangs forever waiting for input,
>> and the menuconfig is sent to the log file.
>
> Indeed, my use-case is to only use it in an already-configured tree.
>
>> Maybe we can bailout if no .config is present.
>
> Hmmm... It's not trivial do detect, because the .config file is not
> necessarily in the current directory, for example:
>
>     brmake -C /pat/to/buil-dir
>
> and I don't want to write a make option interpeter either...
Oh, I see now it is not trivial.

>
> But I'll see what I can do.

Only on the first use the user can get confused.
I think adding a usage example in the file header is enough, saying the script
must be run inside or pointing to (-C from make) an output directory already
configured, or something like that.

[snip]

>> > +    a script that can be run instead of make, that prepends the date in
>> > +    front of each line, redirects all of the build output to a file
>> > +    ("'br.log' in the current directory), and just ouputs the Buildroot
>>
>> Maybe this file will be dropped as discussed in
>> http://patchwork.ozlabs.org/patch/777446/
>> If not, there is a typo
>>
>> ouputs -> outputs
>
> Yup, I already dropped the file locally.
[snip]

The purpose of the script (initially in 00-README.txt, with the typo fixed)
could be placed in the script header as well.

Regards,
Ricardo
_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH 1/4 v2] tools: add a directory to store some useful user-facing tools

Yann E. MORIN-2
Ricardo, All,

On 2017-06-20 23:06 -0300, Ricardo Martincoski spake thusly:

> On Mon, Jun 19, 2017 at 12:42 PM, Yann E. MORIN wrote:
> > On 2017-06-18 22:11 -0300, Ricardo Martincoski spake thusly:
> [snip]
> >> BUT if it is called without a .config file, it hangs forever waiting for input,
> >> and the menuconfig is sent to the log file.
> >
> > Indeed, my use-case is to only use it in an already-configured tree.
> >
> >> Maybe we can bailout if no .config is present.
> >
> > Hmmm... It's not trivial do detect, because the .config file is not
> > necessarily in the current directory, for example:
> >
> >     brmake -C /pat/to/buil-dir
> >
> > and I don't want to write a make option interpeter either...
>
> Oh, I see now it is not trivial.

Yet I think I was able to do it in a minimalist way that should cover
99.999999% of the cases (or so I hope, famous last words):
    https://git.buildroot.org/~ymorin/git/buildroot/commit/?h=yem/contrib&id=221c72431bb12e0dde4051d9317d9fee2bee6eb2

I'll resubmit soon (after a bit more testing...)

> Only on the first use the user can get confused.
> I think adding a usage example in the file header is enough, saying the script
> must be run inside or pointing to (-C from make) an output directory already
> configured, or something like that.
>
> [snip]
> >> > +    a script that can be run instead of make, that prepends the date in
> >> > +    front of each line, redirects all of the build output to a file
> >> > +    ("'br.log' in the current directory), and just ouputs the Buildroot
> >>
> >> Maybe this file will be dropped as discussed in
> >> http://patchwork.ozlabs.org/patch/777446/
> >> If not, there is a typo
> >>
> >> ouputs -> outputs
> >
> > Yup, I already dropped the file locally.
> [snip]
>
> The purpose of the script (initially in 00-README.txt, with the typo fixed)
> could be placed in the script header as well.

Yep, good idea. I'll do that.

Thanks!

Regards,
Yann E. MORIN.

--
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH 1/4 v2] tools: add a directory to store some useful user-facing tools

Arnout Vandecappelle (Essensium/Mind)


On 21-06-17 07:21, Yann E. MORIN wrote:

> Ricardo, All,
>
> On 2017-06-20 23:06 -0300, Ricardo Martincoski spake thusly:
>> On Mon, Jun 19, 2017 at 12:42 PM, Yann E. MORIN wrote:
>>> On 2017-06-18 22:11 -0300, Ricardo Martincoski spake thusly:
>> [snip]
>>>> BUT if it is called without a .config file, it hangs forever waiting for input,
>>>> and the menuconfig is sent to the log file.
>>>
>>> Indeed, my use-case is to only use it in an already-configured tree.
>>>
>>>> Maybe we can bailout if no .config is present.
>>>
>>> Hmmm... It's not trivial do detect, because the .config file is not
>>> necessarily in the current directory, for example:
>>>
>>>     brmake -C /pat/to/buil-dir
>>>
>>> and I don't want to write a make option interpeter either...
>>
>> Oh, I see now it is not trivial.

 How about instead make the script do

make -C ${0%/*}/.. O=${PWD} "${@}"

if $PWD is not the Buildroot dir. This still allows overriding either option on
the command line.

 Not that this solves the issue...


> Yet I think I was able to do it in a minimalist way that should cover
> 99.999999% of the cases (or so I hope, famous last words):
>     https://git.buildroot.org/~ymorin/git/buildroot/commit/?h=yem/contrib&id=221c72431bb12e0dde4051d9317d9fee2bee6eb2

 I haven't tried, but doesn't it do the wrong thing with

make O=... -C ...

?


 But it still doesn't completely solve the issue, because "brmake menuconfig"
still does the wrong thing... Unfortunately I don't know how to solve that
(other than detecting menuconfig as an argument, obviously).

 On a related note, perhaps it's better to redirect stdin in case there is
something interactive during the build (e.g. Qt license approval).


 Regards,
 Arnout

> I'll resubmit soon (after a bit more testing...)
>
>> Only on the first use the user can get confused.
>> I think adding a usage example in the file header is enough, saying the script
>> must be run inside or pointing to (-C from make) an output directory already
>> configured, or something like that.
>>
>> [snip]
>>>>> +    a script that can be run instead of make, that prepends the date in
>>>>> +    front of each line, redirects all of the build output to a file
>>>>> +    ("'br.log' in the current directory), and just ouputs the Buildroot
>>>>
>>>> Maybe this file will be dropped as discussed in
>>>> http://patchwork.ozlabs.org/patch/777446/
>>>> If not, there is a typo
>>>>
>>>> ouputs -> outputs
>>>
>>> Yup, I already dropped the file locally.
>> [snip]
>>
>> The purpose of the script (initially in 00-README.txt, with the typo fixed)
>> could be placed in the script header as well.
>
> Yep, good idea. I'll do that.
>
> Thanks!
>
> Regards,
> Yann E. MORIN.
>

--
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH 0/4 v2] tools: add a directory to contain usefull user-facing tools

Arnout Vandecappelle (Essensium/Mind)
In reply to this post by Yann E. MORIN-2


On 18-06-17 16:23, Yann E. MORIN wrote:

> Thomas, All,
>
> On 2017-06-18 16:15 +0200, Thomas Petazzoni spake thusly:
>> On Sun, 18 Jun 2017 16:01:40 +0200, Yann E. MORIN wrote:
>>> Currently, we provide a few user-facing utilities, like get-developers,
>>> in support/scripts/ . But this directory also contains internal scripts
>>> that a user should not be directly concerned with. Besides, it is
>>> two-level deep in the hierarchy, which is not really nice.
>>>
>>> So, we introduce tools/ as a top-level directory, with the goal to store
>>> and expose all user-facing utilities, while keeping our internal scripts
>>> in support/scripts/ .
>>>
>>> The first patch in the series introduces a new utility, brmake, a
>>> wrapper around make that stores all the output to a log file and
>>> displays only the >>> lines.
>>>
>>> The second and subsequent patches each move a few user-facing utilities
>>> out of support/scripts/ and into tools/ .
>>
>> I am wondering if there is a very clear cut boundary between
>> user-facing tools and non-user facing tools.
>
> Yes, the frontier is not trivial.
>
> I'd say that if a tool is meant to be called by the user, then it
> belongs to tools/. If it is not meant to be alled by a user (even if it
> could be so) then it belongs to support/scripts/.

 Sounds good to me!

 Just to be clear: the main reason why I liked the idea of this directory is
because it makes it easier for users to discover these scripts.

>> For example, graph-build-time or graph-depends are internally used by
>> the Buildroot Makefile, but may also be used directly.
>
> The primary use for those scripts is still an internal use. They are
> supposed to be called from our infra.
>
>> The size-stats
>> tool is also used internally by the Buildroot Makefile, but
>> size-stats-compare is meant to be used directly.
>
> Then the former stays in support/scripts/ while the latter should move
> to tools/.
>
>> What about support/scripts/genimage.sh ?
>
> I would leave it to support/scripts/. It is an internal helper./
> Besides, it needs a few of our variables (TARGET_DIR and BINARIES_DIR),
> so I see it as internal.

 For me, that is a definite indication that it should stay in support/scripts.


 Regards,
 Arnout

>
> Of course, I might not have moved all that can/should be moved. This
> series serves as a base for further discussion.
>
> See the initial submission:
>
>     [PATCH] contrib: add a directory to store "contrib" stuff
>     http://lists.busybox.net/pipermail/buildroot/2016-October/174238.html
>
> Regards,
> Yann E. MORIN.
>
>> Best regards,
>>
>> Thomas
>> --
>> Thomas Petazzoni, CTO, Free Electrons
>> Embedded Linux and Kernel engineering
>> http://free-electrons.com
>

--
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
_______________________________________________
buildroot mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/buildroot
12
Loading...