tcler's blog --- 其实我是一个程序员
Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious.

Use Cases of OCR In Our Test Automation

Here I’d like to introduce you use case of OCR in our test automation:

Background Information

I’m working on Network filesystems(NFS/CIFS/..) testing on linux, and we always
need to install some 3rd party OS in KVM as server or client, but some of them
doesn’t support any mechanism to auto install and configure. e.g:

  • Answer file (unattend.xml)  #for Windows. see: Windows auto install
  • Kickstart file     #for Fedora/CentOS/RHEL. see: kiss-vm -L
  • cloud-init      #for cloud image of many linux distributions. see: kiss-vm -I

and they don’t redirect tty to console and disable sshd by default. this means
I can not automate the install and configuration steps by using expect(1) ..

Should we always install and configure these systems manually in VNC ??!!
does VNC support text mode ? no, see VNC

In computing, Virtual Network Computing (VNC) is a graphical desktop-sharing system that uses the 
Remote Frame Buffer protocol (RFB) to remotely control another computer. 
It transmits the keyboard and mouse events from one computer to another, relaying the **graphical-screen** 
updates back in the other direction, over a network.

OCR debut

Fortunately, I found vncdotool and gocr, after failing to find “VNC text mode”
and after testing and polishing for a while,I got below codes and share to you:

vncget() {
        local _vncaddr=$1

        [[ -z "$_vncaddr" ]] && return 1
        vncdo -s ${_vncaddr} capture $Rundir/_screen.png
        gm convert $Rundir/_screen.png  -threshold 30%  $Rundir/_screen2.png
        gocr -i $Rundir/_screen2.png 2>/dev/null
}

vncput() {
        local vncport=$1
        shift

        which vncdo >/dev/null || {
                echo "{WARN} could not find command 'vncdo'" >&2
                return 1
        }

        [[ -n "$*" ]] && echo -e "\033[1;33m[vncput>$vncport] $*\033[0m"

        local msgArray=()
        for msg; do
                if [[ -n "$msg" ]]; then
                        if [[ "$msg" = key:* ]]; then
                                msgArray+=("$msg")
                        else
                                regex='[~@#$%^&*()_+|}{":?><!]'
                                _msg="${msg#type:}"
                                if [[ "$_msg" =~ $regex ]]; then
                                        while IFS= read -r line; do
                                                [[ "$line" =~ $regex ]] || line="type:$line"
                                                msgArray+=("$line")
                                        done < <(sed -r -e 's;[~!@#$%^&*()_+|}{":?><]+;&\n;g' -e 's;[~!@#$%^&*()_+|}{":?><];\nkey:shift-&;g' <<<"$_msg")
                                else
                                        msgArray+=("$msg")
                                fi
                        fi
                        msgArray+=("")
                else
                        msgArray+=("$msg")
                fi

        done
        for msg in "${msgArray[@]}"; do
                if [[ -n "$msg" ]]; then
                        if [[ "$msg" = key:* ]]; then
                                vncdo -s $vncport key "${msg#key:}"
                        else
                                vncdo -s $vncport type "${msg#type:}"
                        fi
                else
                        sleep 1
                fi
        done
}
vncputln() {
        vncput "$@" "key:enter"
}

ocrgrep() {
        local pattern=$1
        local ignored_charset=${2:-ijkfwe[|:}
        pattern=$(sed "s,[${ignored_charset}],.,g" <<<"${pattern}")
        grep -i "${pattern}"
}

vncwait() {
        local addr=$1
        local pattern="$2"
        local tim=${3:-1}
        local ignored_charset="$4"

        echo -e "\n=> waiting: \033[1;36m$pattern\033[0m prompt ..."
        while true; do vncget $addr | ocrgrep "$pattern" "$ignored_charset" && break; sleep $tim; done
}

projects based on OCR and VNC

ontap simulator in kvm
install freeBSD in kvm article
install freeBSD in kvm code

how to install gocr and vncdo on CentOS/Fedora/RHEL

see: ggv-install.sh


Jianhong