N6TV - Red Pitaya Combine CW & RTTY Skimmer & HDSDR

(These posts originally appeared in the SkimmerTalk reflector, but have been edited and updated with current information and links.  Last updated by N6TV 2017-07-31 23:26 UTC).

From: Bob Wilson, N6TV <[email protected]>
Date: Sun, Jul 23, 2017 at 11:48 PM
Subject: New info. about running Skimmer using the latest 0.97 Red Pitaya OS
To: SkimmerTalk Reflector <[email protected]>

Summary:
  • Ignore my previous warnings.  All Red Pitaya's shipping with OS v0.97 actually do install and run Pavel Demin's SDR Receiver compatible with HPSDR (0.94) just fine using the built-in web installer.  But you must modify /etc/rc.local to get it to start automatically at boot time.
     
  • For RTTY Skimming on Windows 7 or later, ignore "C:\Program Files (x86)\Afreet\RttySkimServ\RttySkimServ.ini" (if any).  Instead, edit "%appdata%\Afreet\Products\RttySkimServ\RttySkimServ.ini" to match center frequencies, segments, and FreqCalibration used by CW Skimmer Server.
     
  • To run HDSDR while skimming, you have to install IPP51 and vcredist_86.exe in addition to CWSL_Tee
     
  • Use a 5V fan to keep it cool
Details:
 
Thanks to direct email help from SDR developer Pavel Demin, I've learned a lot about the Red Pitaya operating systems in the last few days, and I need to update some of the incorrect statements I originally posted about installing the SDR software required to start skimming (see my note of June 30th, 2017, below).  The previously-documented method using Pavel's pre-built "0.95 ecosystem" SD card image works fine, but you will be installing an out-of date and quite limited "BusyBox" Linux operating system that has several limitations, including:
  1. The root password is not retained between power cycles
  2. The clock is not set automatically at power-up
  3. A RAM disk used, so any changes or logs written to the local file system disappear between power cycles
  4. There is no easy way to install useful software tools using the handy "apt-get" command (e.g. apt-get install dos2unix)
  5. The command prompt doesn't display the standard hostname (rp-XXYYZZ) or the current working directory name
In contrast, the current OS shipped with new Red Pitayas (v0.97) is a full-function Ubuntu version 16 operating system, and it does not have any of the limitations listed above.  It automatically sets the clock to UTC, provides a useful command prompt string, and any changes to the root password or file system are always retained across power cycles.
 
Ubuntu seems to be the strategic direction for future Red Pitaya software releases, and some of their apps will require it.  FYI, version "0.96" was yet another Red Pitaya Linux variant OS called "Debian."  It's also full-function, but it seems the Red Pitaya developers have decided to go with Ubuntu instead.
 
The good news is that it is much easier to install the required SDR receiver software now.  Contrary to what I posted at first, Pavel Demin's SDR Receiver compatible with HPSDR (0.94) works quite well with Skimmer on the version 0.97 operating system, and you can easily install and run it using the web page that runs on the box.  What I failed to understand when I first tried that, is that you have to keep your Windows web browser open after you click "Install" and "Run," otherwise the receiver stops running as soon as you close the web page, and Skimmer won't work.
 
Fortunately, there's an easy way to install the right SDR Receiver and have it start automatically at boot time under 0.97, with no need to open a web browser to manually start it, and no need to build a new microSD card.  So here are the steps:
  1. Power up the Red Pitaya using the microSD card image shipped with new units.  If you already wiped it out, you can recreate it by following all the steps here.  I've tried both "Stable" and "Beta" 0.97 operating systems, and they both work.
  2. Determine the IP address assigned to the Red Pitaya.  From Windows, the command ping rp-XXYYZZ may reveal it, where XXYYZZ represents the last six digits of the MAC address printed on the Red Pitaya's Ethernet connector.  Your router will also display the assigned IP address, or the Windows command arp -a may as well.
  3. Open a web browser and go to http://192.168.1.xx (use whatever IP address was assigned to your Red Pitaya by your Router)
  4. Click on the green Application marketplace icon
  5. Scroll down to the SDR Receiver compatible with HPSDR (not the other receiver), and click Install
  6. It should finish installing in less than 30 seconds, then click Run
  7. At this point as long as you don't close the web page that opens after you click Run, you can start your Skimmer Server software and it should find the Red Pitaya (use the 17.7.6 HermesIntf.dll as described in my original post below)
  8. Exit Skimmer Server and close your web browser (in that order).  This will terminate the receiver.
Once installed and tested, to make the SDR Receiver run automatically whenever the Red Pitaya is rebooted, just edit or replace the file named /etc/rc.local on the machine (once).  It needs to have two lines added before the exit 0 command, so the end of the file looks like this
 
# Start the SDR Receiver compatible with HPSDR at boot time
cat /opt/redpitaya/www/apps/sdr_receiver_hpsdr/sdr_receiver_hpsdr.bit >/dev/xdevcfg
. /opt/redpitaya/www/apps/sdr_receiver_hpsdr/start.sh
 
exit 0
 
The dot ( . ) on the 3rd line above is required.
 
Rather than trying to cut and paste this, I've placed a copy of this edited file at http://www.kkn.net/~n6tv/rc.local .  Here's the easiest way to copy it to the Red Pitaya:
  1. Install Putty for Windows and open an SSH connection (like a TELNET connection) to the Red Pitaya's IP address.

    Userid: root
    Password: root
     

  2. You should see a login message at the very top that says:

    Welcome to Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-xilinx armv7l)

    or similar
     

  3. If you haven't done so already, change the root password using the command:

    passwd
     

  4. Enter the following five Unix commands in sequence (updated 2017-07-31):

    cd /etc
    mv rc.local rc.local.old
    wget http://www.kkn.net/~n6tv/rc.local
    chmod +x rc.local
    nano rc.local
         (Verify the file's contents match what I've documented above.  Press Ctrl-X to exit the nano editor.)
    reboot

At this point, the SSH session will terminate, but when the machine reboots, and the "heartbeat" LED is blinking, the SDR receiver will be running, and you'll be able to start skimming using the latest Red Pitaya OS.
 
Warning:  do not open the Application marketplace from the web browser while the SDR receiver is running, otherwise the SDR receiver may halt. From the Ubuntu command prompt you can manually stop the receiver by running:
 
/opt/redpitaya/www/apps/sdr_receiver_hpsdr/stop.sh
 
Finally, I left out one important step when describing how to use HDSDR to listen to the Red Pitaya receiver while it is skimming.  In addition to copying CWSL_Tee.dll to the HDSDR installation directory, and selecting CWSL_Tee on RP-XXYYZZ in Skimmer Server, you must download and install IPP51.
 
The procedure is well documented here:
 
 
The only thing I can't do (yet) on this new OS is monitor the FPGA temperature while the receiver is running.  I found a shell script called xadc.sh that works OK when the receiver is stopped, but it locks up the Red Pitaya immediately if you run it while the SDR Receiver is running, and the only way to recover from that lockup is to power off and on.
 
Update  2017-07-30:  
I've fixed the xadc.sh script so that you can read the CPU temperature without locking up the Red Pitaya.  If running Ubuntu (0.97), you may download and run a copy by signing on to the Red Pitaya and issuing the following commands:
 
wget http://www.kkn.net/~n6tv/xadc.sh
chmod +x xadc.sh
./xadc.sh
 
Result:
 
in_temp0 = 64.84 °C
in_voltage0_vccint = 1.00 V
in_voltage1_vccaux = 1.80 V
in_voltage2_vccbram = 1.00 V
in_voltage3_vccpint = 1.00 V
in_voltage4_vccpaux = 1.80 V
in_voltage5_vccoddr = 1.50 V
in_voltage6_vrefp = 0 V
in_voltage7_vrefn = 0 V
 
Previous temperature measurements returned by xadc_get_value_temp when running the 0.95 image on BusyBox image revealed:
 
Red Pitaya with aluminum "heat sink" enclosure, no fan:
 
~78° C. (scary, 85° C. is max rating)
 
Red Pitaya with aluminum "heat sink" enclosure and a small fan bolted to the top center (I had to drill mounting holes and bend the connector pins on the board to fit the connector):
 
~68° C.
FreqCalibration=1.00000355
 
Red Pitaya in plastic enclosure with original heat sink, and a fan mounted on top of the plastic enclosure:
 
~62° C.
 
Red Pitaya in plastic enclosure with fan mounted inside the enclosure, with one corner crew touching the heat sink:
 
~58° C.
FreqCalibration=1
 
The last solution is best, but it is also the noisiest.  I used two small 4-40 Nylon nuts and bolts to mount the 30 mm x 30 mm x 10 mm 5 VDC fan. The positive connector pin is the one closest to the center of the board. 
 
Update 2017-07-30:  If you're still running the 0.95 Linux BusyBox image, and you want to check the FPGA temperature:
  1. Download

    https://sites.google.com/site/hansdorn/pics/xadc.tgz

    using a web browser (the wget commmand doesn't work)
     

  2. Copy xadc.tgz to the Red Pitaya.  On Windows (when Putty installed), use the command:

    pscp -p xadc.tgz [email protected]:/tmp

    (Substitute the IP address of your RP.  Command will prompt for the root password)
     

  3. Open a putty session to the Red Pitaya, then issue the following commands, in order:

    cd /tmp
    gunzip xadc.tgz
    tar xvf xadc.tar
    export PATH="$PATH$:/tmp/xadc"
    xadc_get_value_temp

73,
Bob, N6TV

From: Bob Wilson, N6TV <[email protected]>
Date: Fri, Jun 30, 2017 at 10:40 PM
Subject: CW Skimmer + RTTY Skimmer running simultaneously on Red Pitaya - Success!
To: SkimmerTalk Reflector <[email protected]>

I finally succeeded in getting both CW and RTTY Skimmer to run simultaneously on a single Red Pitaya SDR.  I hope someone else can confirm that the procedure below works for them too.
 
Furthermore, I am able to view a live waterfall display and listen to any band being monitored by the same SDR at the same time, using the HDSDR software.  And finally, I was able to configure the RBNAggregator to launch both skimmers automatically and in the right sequence.
 
N6TV_RedPitaya_CW_and_RTTY_Skimmer_and_HDSDR.png
 
I'm not sure if the procedure for doing this on the Red Pitaya has been well-documented yet, so here's what to do.  It's similar to the process for the QS1R, with some very important differences.
 
Red Pitaya Hardware and SDR Setup:
  1. The only Red Pitaya OS version that worked for me was the Customized SD Card Image linked below (Update 2017-07-30: see new info. above).  Do not waste time trying any other versions, such as the one that comes pre-installed on the Red Pitaya micro SD card, even though it's newer, or version 0.94 on the Red Pitaya web site (Update 2017-07-30:  these are acutally OK)
  2. As discovered by K1TTT, use a single jumper to bypass the 1M ohm input impedance circuit.  Connect pins 2 and 5 (the two middle pins) together (pictures in W2NAF's doc linked below)
  3. I'm using a tiny 14:1 transformer supplied by Red Pitaya on the input.  At this time they are not planning to sell it separately; it will only be supplied with their new transceiver kit. It is a MiniCircuits model T14-1+ mounted on a small PC board with male and female SMA connectors.
  4. I found a 10 dB preamp to be helpful on the high bands with my antenna.  Without it, I saw no change in noise floor when I disconnected the antenna on the high bands.  It was very nice to watch the noise floor in HDSDR.
  5. Configuration:  VE3DO Loop Antenna -> DX Engineering Receiver Guard -> 10 dB Pre-Amp -> 14:1 transformer -> Red Pitaya Input 1.
  6. I recommend a short BNC to SMA pigtail, rather than a BNC to SMA adapter, to reduce strain on the SMA connector.  One example is the JetStream JT4130J3 sold by GigaParts.
Quick Summary, for Windows (software links follow):
  1. Copy CWSL_Tee.dll to the SkimSrv installation directory
  2. Copy CWSL_Tee.dll to the RttySkimServ installation directory
  3. Copy HermesIntf.dll to the SkimSrv installation directory
  4. Do NOT (repeat, do NOT) copy HermesIntf.dll to the RttySkimServ directory, otherwise everything stops when you try to start RTTY Skimmer Server after CW Skimmer Server
  5. Create a two-line file named CWSL_Tee.cfg containing only these two lines and nothing more:

    HermesIntf
    64
     

  6. Save CWSL_Tee.cfg in the SkimSrv directory
  7. Copy CWSL_Tee.cfg to the RTTYSkimServ directory
  8. Copy CWSL_Tee.cfg to the RBN Aggregator directory (a very important step)
  9. Copy ExtIO_CWSL.dll to the HDSDR installation directory, nothing more
  10. Just as for the QS1R, configure both SkimSrv/SkimSrv.ini and RttySkimServ/RTTYSkimServ.ini with identical values for the following lines:
    (mostly left blank below, for clarity):
    DeviceName=01 CWSL_Tee on
    FreqCalibration=1
    CenterFreqs48=
    CenterFreqs96=
    CenterFreqs192=
    SegmentSel48=
    SegmentSel96=
    SegmentSel192=
    Rate=1
  11. In SkimSrv.ini use:
    [Telnet]
    Port=7300
    In RTTYSkimServ.ini use:
    [Telnet]
    Port=7301
    (value not critical, but the two skimmers must not be using the same TELNET port)
  12. Manually start CW Skimmer Server first
  13. If necessary, select the SDR named "01 CWSL_Tee on ..." and verify normal operation. If the device name displays strange characters, it means you are using an outdated version of HermesIntf.dll (a fix was made available by K3IT on 2017-07-06)
  14. Manually start RTTY Skimmer Server second
  15. Verify that both skimmers are still running and show no errors or warning messages.
    If you get errors, note them and immediately terminate both skimmer programs.
    Look at the CWSL_Tee and HermesIntf log files written to both directories to help identify the problem
  16. (optional) Start HDSDR and select ExtIO_CWSL.dll when prompted
  17. In HDSDR, use only the small pull-down menu in the ExtIO pop-up dialog to change bands.  You can listen to any band being monitored by the Skimmers.  Do not change the LO frequency, only the TUNE frequency (just click on the spectrum or waterfall)
Once the above is verified as operational, it's time to test the process via the RBN Aggregator.
  1. Terminate HDSDR, RTTY Skimmer Server, and CW Skimmer Server, in that order
  2. Launch the RBN Aggregator and quickly click on the ini Files tab and select Stop Rotation
  3. Check the path and file names make sure you're launching SkimSrv.exe, and the same SkimSrv.ini file edited previously.  View the file with the Edit button to verify.
  4. On the Secondary Skimmers tab, last slot (#8), select IP address 127.0.0.1 port 7301 or whatever you used in RttySkimServ.ini
  5. On the RttySkimServ tab, verify the full path to RttySkimServ.exe and RttySkimServ.ini - do not use any other file name here.
  6. Check the box Force RTTY Skimmer Server to follow CW Skimmer Server
  7. Go back to the ini Files tab and click the NOW button next to the proper SkimSrv.ini to launch both skimmers, in order.
  8. Verify that both Skimmer Servers start up normally with no errors, and show one telnet connection each.
  9. In the Skimmer Traffic tab of the Aggregator, you should see messages indicating telnet connections to both CW Skimmer and RTTY Skimmer
  10. In the Status tab, there should be no errors
  11. If you do have errors, look for HermesIntf and CWSL_Tee log files in the Aggregator directory.  The most common problem is not having the correct CWSL_Tee.cfg file in the Aggregator directory.
Here are useful links to all the required software files and extras, with proper credit to all the developers and enthusiasts who made it possible for simple folks like us to replace our failed QS1Rs with a Red Pitaya or two:
Random notes on the Red Pitaya:
  1. We all owe Pavel Demin a big THANK YOU for his Red Pitaya SDR Receiver.
  2. Follow the steps in W2NAF's excellent Red Pitaya RBN How-To Guide exactly; don't take any shortcuts:
    • The Red Pitaya Starter kit includes a new and improved v0.97 operating system, which boots fine and has handy features, but I have not been able to find a binary version of Pavel Demin's HPSDR code that runs on it (Update: 2017-07-30 - it works fine, see note of July 23rd above).
    • For now, use only the v0.95 ecosystem Zip image linked above (Update 2017-07-30 - the v0.97 Ubuntu image is now preferred).
    • Insert the micro SD card and adapter included with the Red Pitaya Start Kit into your Windows computer, format the FAT partition (wiping out v0.97), then extract Pavel Demin's entire SD Card Image to the root directory.
    • You can get back to a vanilla v0.97 later using images on the Red Pitaya web site .
    • Leave the other hidden partitions on the 4GB micro SD card alone, just format the FAT partition, do not delete or touch any other partitions.
    • Extracting the Zip without first formatting the partition does not work.
    • Installing the v0.94 SDR receiver from the "Marketplace" link instead of from the Zip image does not work.
    • You don't have to use PowerSDR, which I found very hard to use.  HDSDR seems much more intuitive to me.
    • You do not need to install Bonjour to access the Red Pitaya box from Windows; use the Windows command arp -a or your Router interface to find the IP address assigned to the Red Pitaya, then open your browser and enter that IP address as a URL
  3. Just use FreqCalibration=1 in the Skimmer .ini files.  I wasted a lot of time trying to get just the right calibration setting on the highest band, but then it messed up the other bands. Frequency calibration is not consistently "high" or "low" from band to band due varying sampling conversions, and since the Skimmer provides only one calibration setting, it's best to leave it alone.  (Update 2017-07-30:  once the temperature is stabilized with a fan, the FreqCalibration procedure works well.  Set the value to Displayed Frequency / True Frequency, with 8 digits after the decimal point.)
  4. Frequency accuracy got better when I added a small 30mm fan via the plastic case, as recommended.  I did have to crimp on my own 0.1 mm fan connector.  Not well indicated on the board is that the positive pin of the fan connector is the pin closest to the center of the board.
  5. You can calibrate HDSDR very accurately using Options | Calibration Settings while tuned to WWV in EECS (AM) mode with AFC enabled.
  6. With OmniRig support, HDSDR can follow your transceiver's VFO and vice versa (Options -> CAT to radio (Omni-Rig) -> sync from Omni-rig (enable), sync to Omni-Rig (enable)
  7. Use Putty on Windows to get Unix command line access to the Red Pityaa (userid root, password root).
  8. Use the Unix command rdate -sp wwv.nist.gov to accurately set the internal Red Pitaya clock (in UTC) after power up.  Any error logs produced will then be properly time-stamped.  Change the password first, then set the clock.  It may not be possible to automate setting the clock via NTP on v0.95.
  9. Use the passwd command to change the root password.  Unfortunately, the new password is not saved between power cycles in this version of the OS, so it must be reset every time you power up the Red Pitaya. It seems this version uses a RAMDISK, so changes made to local files are not saved between power cycles. :-(
  10. The top command will display the Red Pitaya CPU utilization and memory use.  I never saw the CPU usage exceed 25% when skimming 6 bands at 192 kHz.  Type q to exit and return to the command line.
  11. Type exit to exit from the Unix command line shell
  12. If you open the IP address of the Red Pitaya in your browser, you'll see a nice friendly interface with an "Application Marketplace" link, but that software is not pre-installed on the SD card, it just redirects to http://bazaar.redpitaya.com/, and the SDR Receiver v0.94 on that site is old and does not run at all on v0.95.  DO NOT CLICK INSTALL!  (Update 2017-07-30 -- this actually works fine; I misunderstood that you have to click Install, Run, and keep the next page open to run the SDR receiver manually).
  13. Despite W4KAZ's blog post (http://w4kaz.com/qth/?p=2786), the only SDR receiver build that seems to work with CW Skimmer Server and RTTY Skimmer Server at this time is 0.95 pre-built version.  I think there are still outstanding issues preventing the SDR code from running on any new version of the OS.  (Update 2017-07-30:  all those problems were fixed).
  14. I tried SDR-Console and WINRAD but they are not compatible.  Only HDSDR and PowerSDR worked for me, and only HDSDR works when the Skimmers are running.
  15. The monitor -ams command is documented to display Red Pitaya CPU temperature, but it isn't implemented in v0.95. :-(
  16. Error messages in the Red Pitaya are saved in in /tmp/log/redpitaya_nginx (use the tab key to auto-complete long file names)
  17. Use nano filename to view a log file in a full screen Unix text editor
  18. Press Ctrl-X to exit the editor
73,
Bob, N6TV