Issues in installing NVIDIA 390/396 driver on Ubuntu 18.04

Met some strange problems when following this instruction to install NVIDIA-396 driver on Ubuntu 18.04 (kernel 4.15.0-33-generic x86_64). It tells the driver is successfully installed. But when I run:

nvidia-smi

to check the status. It tells me:

NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

Tried another command:

lsmod | grep nvidia

Nothing is printed out. Definitely, something is wrong. I remove the installed drivers using:

sudo apt remove –purge *nvidia*

Rerun the installation command:

sudo ubuntu-drivers autoinstall

Finally, I found in the log that the installation encountered an error building event :

Error! Bad return status for module build on kernel: 4.15.0-33-generic (x86_64)

Apt suggests to check the make file /var/lib/dkms/nvidia/396.54/build/make.log for details. The tail 300 lines of this file is as follows:

cc -Wp,-MD,/var/lib/dkms/nvidia/396.54/build/nvidia/.nv-frontend.o.d -nostdinc -isystem /usr/lib/gcc/x86_64-linux -gnu/4.8/include -I./arch/x86/include -I./arch/x86/include/generated -I./include -I./arch/x86/include/uapi -I./arc h/x86/include/generated/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/kconfig.h -Iubuntu /include -D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wch ar -Werror-implicit-function-declaration -Wno-format-security -std=gnu89 -fno-PIE -mno-sse -mno-mmx -mno-sse2 -mno-3 dnow -mno-avx -m64 -falign-jumps=1 -falign-loops=1 -mno-80387 -mno-fp-ret-in-387 -mpreferred-stack-boundary=3 -mtune =generic -mno-red-zone -mcmodel=kernel -funit-at-a-time -DCONFIG_X86_X32_ABI -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNA L_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_SSSE3=1 -DCONFIG_AS_CRC32=1 -DCONFIG_AS_AVX=1 -DCONFIG_AS_AVX2=1 -DCONFIG_AS_AVX512=1 -DCONFIG_AS_SHA1_NI=1 -DCONFIG_AS_SHA256_NI=1 -pipe -Wno-sign-compare -fno- asynchronous-unwind-tables -mindirect-branch=thunk-extern -mindirect-branch-register -DRETPOLINE -fno-delete-null-po inter-checks -O2 -Wno-maybe-uninitialized –param=allow-store-data-races=0 -DCC_HAVE_ASM_GOTO -Wframe-larger-than=10 24 -fstack-protector-strong -fno-omit-frame-pointer -fno-optimize-sibling-calls -pg -I/var/lib/dkms/nvidia/396.54/bu ild/common/inc -I/var/lib/dkms/nvidia/396.54/build -Wall -MD -Wno-cast-qual -Wno-error -Wno-format-extra-args -D__KE RNEL__ -DMODULE -DNVRM -DNV_VERSION_STRING=\”396.54\” -Wno-unused-function -Wuninitialized -fno-strict-aliasing -mno -red-zone -mcmodel=kernel -DNV_UVM_ENABLE -I/var/lib/dkms/nvidia/396.54/build/nvidia -DNV_BUILD_MODULE_INSTANCES=0 – DNVIDIA_UNDEF_LEGACY_BIT_MACROS -UDEBUG -U_DEBUG -DNDEBUG -DMODULE -DKBUILD_BASENAME='”nv_frontend”‘ -DKBUILD_MOD NAME='”nvidia”‘ -c -o /var/lib/dkms/nvidia/396.54/build/nvidia/nv-frontend.o /var/lib/dkms/nvidia/396.54/build/nvidi a/nv-frontend.c
cc -Wp,-MD,/var/lib/dkms/nvidia/396.54/build/nvidia/.nv-instance.o.d -nostdinc -isystem /usr/lib/gcc/x86_64-linux -gnu/4.8/include -I./arch/x86/include -I./arch/x86/include/generated -I./include -I./arch/x86/include/uapi -I./arc h/x86/include/generated/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/kconfig.h -Iubuntu /include -D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wch ar -Werror-implicit-function-declaration -Wno-format-security -std=gnu89 -fno-PIE -mno-sse -mno-mmx -mno-sse2 -mno-3 dnow -mno-avx -m64 -falign-jumps=1 -falign-loops=1 -mno-80387 -mno-fp-ret-in-387 -mpreferred-stack-boundary=3 -mtune =generic -mno-red-zone -mcmodel=kernel -funit-at-a-time -DCONFIG_X86_X32_ABI -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNA L_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_SSSE3=1 -DCONFIG_AS_CRC32=1 -DCONFIG_AS_AVX=1 -DCONFIG_AS_AVX2=1 -DCONFIG_AS_AVX512=1 -DCONFIG_AS_SHA1_NI=1 -DCONFIG_AS_SHA256_NI=1 -pipe -Wno-sign-compare -fno- asynchronous-unwind-tables -mindirect-branch=thunk-extern -mindirect-branch-register -DRETPOLINE -fno-delete-null-po inter-checks -O2 -Wno-maybe-uninitialized –param=allow-store-data-races=0 -DCC_HAVE_ASM_GOTO -Wframe-larger-than=10 24 -fstack-protector-strong -fno-omit-frame-pointer -fno-optimize-sibling-calls -pg -I/var/lib/dkms/nvidia/396.54/bu ild/common/inc -I/var/lib/dkms/nvidia/396.54/build -Wall -MD -Wno-cast-qual -Wno-error -Wno-format-extra-args -D__KE RNEL__ -DMODULE -DNVRM -DNV_VERSION_STRING=\”396.54\” -Wno-unused-function -Wuninitialized -fno-strict-aliasing -mno -red-zone -mcmodel=kernel -DNV_UVM_ENABLE -I/var/lib/dkms/nvidia/396.54/build/nvidia -DNV_BUILD_MODULE_INSTANCES=0 – DNVIDIA_UNDEF_LEGACY_BIT_MACROS -UDEBUG -U_DEBUG -DNDEBUG -DMODULE -DKBUILD_BASENAME='”nv_instance”‘ -DKBUILD_MOD NAME='”nvidia”‘ -c -o /var/lib/dkms/nvidia/396.54/build/nvidia/nv-instance.o /var/lib/dkms/nvidia/396.54/build/nvidi a/nv-instance.c
cc: error: unrecognized command line option ‘-fstack-protector-strong’
scripts/Makefile.build:332: recipe for target ‘/var/lib/dkms/nvidia/396.54/build/nvidia/nv-frontend.o’ failed
make[2]: *** [/var/lib/dkms/nvidia/396.54/build/nvidia/nv-frontend.o] Error 1
make[2]: *** Waiting for unfinished jobs….
cc -Wp,-MD,/var/lib/dkms/nvidia/396.54/build/nvidia/.nv.o.d -nostdinc -isystem /usr/lib/gcc/x86_64-linux-gnu/4.8/ include -I./arch/x86/include -I./arch/x86/include/generated -I./include -I./arch/x86/include/uapi -I./arch/x86/inc lude/generated/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/kconfig.h -Iubuntu/include -D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -Werro r-implicit-function-declaration -Wno-format-security -std=gnu89 -fno-PIE -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno -avx -m64 -falign-jumps=1 -falign-loops=1 -mno-80387 -mno-fp-ret-in-387 -mpreferred-stack-boundary=3 -mtune=generic -mno-red-zone -mcmodel=kernel -funit-at-a-time -DCONFIG_X86_X32_ABI -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_SSSE3=1 -DCONFIG_AS_CRC32=1 -DCONFIG_AS_AVX=1 -DCONFIG _AS_AVX2=1 -DCONFIG_AS_AVX512=1 -DCONFIG_AS_SHA1_NI=1 -DCONFIG_AS_SHA256_NI=1 -pipe -Wno-sign-compare -fno-asynchron ous-unwind-tables -mindirect-branch=thunk-extern -mindirect-branch-register -DRETPOLINE -fno-delete-null-pointer-che cks -O2 -Wno-maybe-uninitialized –param=allow-store-data-races=0 -DCC_HAVE_ASM_GOTO -Wframe-larger-than=1024 -fstac k-protector-strong -fno-omit-frame-pointer -fno-optimize-sibling-calls -pg -I/var/lib/dkms/nvidia/396.54/build/commo n/inc -I/var/lib/dkms/nvidia/396.54/build -Wall -MD -Wno-cast-qual -Wno-error -Wno-format-extra-args -D__KERNEL__ -D MODULE -DNVRM -DNV_VERSION_STRING=\”396.54\” -Wno-unused-function -Wuninitialized -fno-strict-aliasing -mno-red-zone -mcmodel=kernel -DNV_UVM_ENABLE -I/var/lib/dkms/nvidia/396.54/build/nvidia -DNV_BUILD_MODULE_INSTANCES=0 -DNVIDIA_U NDEF_LEGACY_BIT_MACROS -UDEBUG -U_DEBUG -DNDEBUG -DMODULE -DKBUILD_BASENAME='”nv”‘ -DKBUILD_MODNAME='”nvidia”‘ -c -o /var/lib/dkms/nvidia/396.54/build/nvidia/nv.o /var/lib/dkms/nvidia/396.54/build/nvidia/nv.c
cc: error: unrecognized command line option ‘-fstack-protector-strong’
scripts/Makefile.build:332: recipe for target ‘/var/lib/dkms/nvidia/396.54/build/nvidia/nv-instance.o’ failed
make[2]: *** [/var/lib/dkms/nvidia/396.54/build/nvidia/nv-instance.o] Error 1
cc: error: unrecognized command line option ‘-fstack-protector-strong’
scripts/Makefile.build:332: recipe for target ‘/var/lib/dkms/nvidia/396.54/build/nvidia/nv.o’ failed
make[2]: *** [/var/lib/dkms/nvidia/396.54/build/nvidia/nv.o] Error 1
Makefile:1552: recipe for target ‘_module_/var/lib/dkms/nvidia/396.54/build’ failed
make[1]: *** [_module_/var/lib/dkms/nvidia/396.54/build] Error 2
make[1]: Leaving directory ‘/usr/src/linux-headers-4.15.0-33-generic’
Makefile:79: recipe for target ‘modules’ failed
make: *** [modules] Error 2

What is “‘-fstack-protector-strong’“? My instinct tells me it is related to the gcc or g++. After searching in Google, I decide to upgrade the current gcc/g++ from 4.8 to 4.9. Since 4.9 is not available in Ubuntu 18.04 repository yet, I temporarily add the 16.04 repository to the apt source list.

First, add xenial to /etc/apt/sources.list temporarily : sudo gedit /etc/apt/sources.list

deb http://dk.archive.ubuntu.com/ubuntu/ xenial main
deb http://dk.archive.ubuntu.com/ubuntu/ xenial universe

and do sudo apt update && sudo apt install g++-4.9See stackflow for details.

Remember to remove the xenial from the sources.list after the installation is done.

Then, remove the old NVIDIA driver

sudo apt remove –purge *nvidia*

and re-install it.

sudo ubuntu-drivers autoinstall

Everything looks fine. Try nvidia-smi, the output is correct this time:

WeChat Screenshot_20180823122141

How to convert PDF to Word

This will be very short. One word: Word. Use MicroSoft Word 2016 to open your PDF file and save as a word docx file.

Don’t be scammed to download all those junk and fake-open-source software like “UniPDF or Nemo or Free PDF-to-Word Converter” from Internet. Don’t install any software from any unknown websites, especially those with all sorts of ads posting everywhere. 99% of their software are virus.

How to install Cloudstack to Ubuntu 18.04 Bionic Beaver

It would be a disaster if you want to install Cloudstack onto Ubuntu 18.04 at this time (one month after its release). All the dependencies are running into incompatible issues and making it impossible to do it. However, in my situation I have to do this right now. I have read all the posts on the web, including Rohit’s blog which is not helping. Thank Robit anyway as I finally found someone facing the same problem.

But I told myself that now is 2018 and no technical issue is unsolvable. The answer is actually simple: docker. Install docker onto Ubuntu 18.04 and install cloudstack inside docker container. Use docker container as either a cloudstack management or a cloudstack agent, according to your choice.

Dell inspiron 15 7000 gaming laptop automatically shutdown after a while

I bought a new Inspiron (honestly, open box at a very good price) and want to run deep learning program on it. I started the program before taking off and left it running for the whole night. When coming back in the morning, it really shocked me when I found it was mysteriously shut down. What the hell? I instantly googled everything about this automatically-shut-down thing and tried all the solutions, but the problem still persists after another night. I almost returned it at that time, but it is a very nice deal and looks really stunning. Will feel very bad since I already spend two days on it and set up the software environment. So I move on to find solutions.

Finally. Long story short, go to the control panel -> power options ->edit plan settings -> change advanced power settings -> turn off hard disk after (set it to never). Problem gone. That’s it.

Deeplearning4j CPU MKL DLL problem

Recently, I am excising the DeepLearning4j library which is developing rapidly with a growing user community. I use it because Java is my
“mother” language.

I have cloned the dl4j-examples and run without problem for nearly a year. In the last two months, I leave it behind and focus on something else (and installed some software for the new tasks). Yesterday, when I rerun the dl4j-examples, the program exited with a message “Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll”. I searched all over the web and tried to compile through the ND4j. But, nothing helps.

After reading this page, I understand DL4j can use either MKL or OpenBLAS. I never installed MKL and always use OpenBLAS which is a built-in module of ND4j (as I understand). Why does ND4j suddenly choose MKL as its blas vendor instead of OpenBLAS?

Eventually, my solution is checking the PATH environment variable. My case is on Windows. Go to the environment variable setting page and examine the PATH variable carefully. Remove the pathes which may cheat DL4j to believe your machine has MKL installed. After the change, make sure reboot your machine to let the change take effect. The JVM won’t adopt the new changes until it is restarted.

In my case, I removed the pathes which are appearently appended to the PATH variable after I installed new software. The problem is solved.

Here is the log of OpenBLAS:

 jar:file:/C:/Users/Administrator/.m2/repository/org/nd4j/nd4j-native/0.9.1/nd4j-native-0.9.1.jar!/
 15:41:16.342 [main] INFO org.reflections.Reflections - Reflections took 890 ms to scan 13 urls, producing 31 keys and 227 values
 15:41:26.898 [main] INFO org.nd4j.nativeblas.Nd4jBlas - Number of threads used for BLAS: 2
 15:41:26.995 [main] INFO o.n.l.a.o.e.DefaultOpExecutioner - Backend used: [CPU]; OS: [Windows 7]
 15:41:26.995 [main] INFO o.n.l.a.o.e.DefaultOpExecutioner - Cores: [4]; Memory: [1.8GB];
 15:41:26.996 [main] INFO o.n.l.a.o.e.DefaultOpExecutioner - Blas vendor: [OPENBLAS]
 15:41:28.658 [main] DEBUG org.reflections.Reflections - going to scan these urls:
 file:/C:/Users/Administrator/.m2/repository/org/slf4j/jul-to-slf4j/1.7.12/jul-to-slf4j-1.7.12.jar

Using Long Short-Term Memory Recurrent Neural Network in Land Cover Classification on Landsat and Cropland Data Layer time series

Land cover maps are significant in the agricultural analysis. However, the existing workflow of producing maps takes too long. This work builds long short-term memory (LSTM) recurrent neural network (RNN) model to improve the update frequency. An end-to-end framework is proposed to train the model. Landsat scenes are used as Earth observations. Field-measured data and CDL (Cropland Data Layer) are used as ground truth. The network is deeply trained using state-of-the-art techniques. Finally, we tested the network on multiple Landsat images to produce five-class land cover maps with timestamps. The results are visualized and compared with CDL and ground truth. The experiment shows a satisfactory overall accuracy (>97%) and proves the feasibility of the model. This study paves a path to efficiently using LSTM RNN in remote sensing image classification.

 

GeoFairy.Map wins in NGA Disparate Data Competition

geofairy

GeoFairy.Map, derived from GeoFairy, is one of the final winners in Stage 1 of NGA Disparate Data Competition.

Geofairy.Map is a system created as an easy-to-go entry for the widely disparate datasets faced by both governments and corporations today. The system hubs many kinds of data sources, especially governmental datasets from NGA, NOAA, NASA, USGS, EPA, etc, and makes it much easier for people to discover, access, view and retrieve them via as few clicks as possible. With Geofairy.Map, users no longer need to jump about on the Internet and get exhausted to reach the information they are interested in. The time spent by users on data accessing is significantly reduced. This feature will be very helpful in the situations of emergency such as military actions, disaster responding and daily information quick inquiry. The datasets in Geofairy.Map are very heterogeneous in formats such as WMS, WMTS, ArcGIS REST, GeoJSON, Geopackage, ESRIShapefile, KML, NITF, GeoTiff, CSV, XLS, XML and GML. Geofairy.Map managed to manipulate them in a very uniformed, intuitive and secured interface. Geofairy.Map has both web based and mobile based clients. Everytime users submit any request, Geofairy.Map will repeatedly perform security checking on client, networking and server. The operation can be only processed when a clearance is granted for the current communication. Among the checking, client identity and networking anti-hijacking are the two major items that have to be validated. We designed and are making Geofairy.Map to take advantage of the Apple iOS Finger Print system to identify the users. On the backstage, as many data sources have restrictions on data access, we perform the server-to-server communication following secure channels and policies such as HTTPS, HTTP Auth, Token and WS-security policies.

The datasets already integrated in Geofairy.Map include (both NGA and other public datasets):

  • Hurricane_Hermine
  • Disaster Response – NEPAL
  • Wildlife Trafficking
  • Geonames – foreign
  • Navy BlueMarble – low res (WMS)
  • Navy BlueMarble – low res (WMTS)
  • Sample NITF Data
  • Sample Landsat Data
  • FAA Airports
  • Global Ground Control Point Dataset
  • AsiaMaritime Transparency (Public service)
  • Bathymetric, GeophysicalMaps (NOAA)
  • West Africa Ebola Outbreak
  • Data for World Food Program
  • S. Fire Data
  • NOAA Historical Hazard
  • Global Weather Map (from OpenWeatherMap)
  • USDA Land Cover Product (Crop Data Layer)
  • USDA National Hardness Zone Map
  • USGS DEM 90 Meters
  • NASA AIRS Product
  • NASA AIRS NRT Product
  • NASA TRMM Product
  • NASA OMI Product
  • Vegetation Monitoring Product from VegScape

A demo video of the web page version of Geofairy.Map is here: https://youtu.be/hZdZacJAjrw

A demo video of the mobile App version of Geofairy.Map is here: https://youtu.be/dIGi0xvvY0s

Great thanks to the authors of GeoFairy.Map (order by contribution from high to low):

Ziheng Sun (zsun@gmu.edu)

Liping Di (ldi@gmu.edu)

Gil Heo

Chen Zhang

Ziao Liu

Hui Fang

Liping Guo

©All rights are reserved by authors.