Skip to content

BrainWave Net is a high performance research Abstract Summarizer web application that is created on various testedNLP deep learning models. This was solely created for the purpose such that research scientists and common folksfrom various fields can get their research abstracts summarized in a very easy to read and understandable format

Notifications You must be signed in to change notification settings

Purbayan2014/Using-BERT-Models-for-Sequential-Text-Classification-in-Biomedical-Abstracts

Repository files navigation

Sequential-text-classification-using-deep-sequence-modelling [BrainWave Net]

BrainWave Net is a high performance research Abstract Summarizer web application that is created on various testedNLP deep learning models. This was solely created for the purpose such that research scientists and common folksfrom various fields can get their research abstracts summarized in a very easy to read and understandable format

Demo

The frontend and the rest api backend code are in private repos if you want to check them out send me a mail to [email protected]

BrainWaveNet-2023-03-28_15.58.15.mp4

Troubleshooting

LibCublas Issue

nvidia-cublas

image

curl https://developer.download.nvidia.com/hpc-sdk/ubuntu/DEB-GPG-KEY-NVIDIA-HPC-SDK | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-hpcsdk-archive-keyring.gpg

echo 'deb [signed-by=/usr/share/keyrings/nvidia-hpcsdk-archive-keyring.gpg] https://developer.download.nvidia.com/hpc-sdk/ubuntu/amd64 /' | sudo tee /etc/apt/sources.list.d/nvhpc.list

sudo apt-get update -y

sudo apt-get install -y nvhpc-22-9

For Tensorflow 2.x

python3

import tensorflow as tf

config = tf.compat.v1.ConfigProto()

config.gpu_options.allow_growth = True

session = tf.compat.v1.Session(config=config)

For Tensorflow 1.x

config = tf.ConfigProto()

config.gpu_options.allow_growth = True

session = tf.Session(config=config....)

Cuda CUL-INT Secure Boot Issue

image

Install the appropriate BIOS version
Disable Secure Boot in BIOS settings

import tensorflow as tf

tf.config.list_physical_devices("GPU")

[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

Pytorch Cuda Visible device Issue

CUDA initialization: CUDA unknown error - this may be due to an incorrectly set up environment, e.g. changing env variable CUDA_VISIBLE_DEVICES after program start

sudo rmmod nvidia_uvm

sudo modprobe nvidia_uvm

Nvidia-NUMA node connection Issue

Non-uniform memory access (NUMA) systems are server platforms with more than one system bus. These platforms can utilize multiple processors on a single motherboard, and all processors can access all the memory on the board. When a processor accesses memory that does not lie within its own node (remote memory), data must be transferred over the NUMA connection at a rate that is slower than it would be when accessing local memory. Thus, memory access times are not uniform and depend on the location (proximity) of the memory and the node from which it is accessed.

Check Numa Node Connection

cat /sys/bus/pci/devices/0000\:01\:00.0/numa_node
-1
-1 means no connection, 0 means connected.

Reattach Numa node connection

sudo echo 0 | sudo tee -a /sys/bus/pci/devices/0000\:01\:00.0/numa_node
0

Nvidia Cuda Libraries Missing

Install the approprita Cudatoolkit and CudaDNN from Nvidia Official Site or using conda as below
conda install -c conda-forge cudnn=8.1.0 cudatoolkit=11.7

Export the libraries into your current enviornment
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/
or Automate this process everytime so that it is included everytime a conda env is invoked
mkdir -p $CONDA_PREFIX/etc/conda/activate.d echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/' > $CONDA_PREFIX/etc/conda/activate.d/env_vars.sh

About

BrainWave Net is a high performance research Abstract Summarizer web application that is created on various testedNLP deep learning models. This was solely created for the purpose such that research scientists and common folksfrom various fields can get their research abstracts summarized in a very easy to read and understandable format

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 3

  •  
  •  
  •