I have been studying in Japan for more than five years in completing my master’s and doctoral degrees. For me, I was used to living in Japan including daily, research, and social life. Yet, I still found it difficult when I came to job hunting in Japan. Lack of information, unfamiliar with the job application process, period of application, the skill of writing resume and the documents required, these reasons resulting from the job hunting in Japan is tough for me. From the experience of a friend, I got to know ASIA to JAPAN. Through ASIA to JAPAN, she was able to get job offers within just a few days. This motivates me and I decides to give it a try. The application is simple by just submitting the required documents of basic information and a short video clip for self-introduction. In the past years, after the screening, an invitation to Japan for interviews will be offered with free transportation and accommodation once you have been selected by 3 or more companies. During my submission, due to the Covid, the situation to have a face-to-face interview is crucial. Everything goes online. Well, this doesn’t mean something bad. This turns out that more opportunities for us to have interviews, as you will be called for the interview even just selected by 1 company instead of 3, and the selection is done more often. Once you have been selected for interviews, the company details, the job description will be provided so that you can decide the level of interest after the survey to the company. The common expected question will be asked in the interview, what kind of people the company looking for, and other information will be provided as well. This helps me in understanding the company and decide whether interest in the company/job. On top of that, the staff of ASIA to JAPAN will arrange the practice and mock interview which helps a lot. They are friendly and helpful, providing further information upon query, pointing out the weakness of your performance, and providing guidance for it. I joined the program twice. In the first year, I was selected for interviews by B. Company, G. Company, and R. company, and I got a job offer from R. company. However, due to some circumstances, I had to turn down the job offer. In the following year, I joined again this program and I got an offer from A. company. Other than joining this program, I tried to apply to some other companies directly through their official website, yet the process, screening is tough, especially due to the Covid situation. I am grateful in getting a job offer that matches my interest job field and education skill through ASIA to JAPAN. If you are looking for a job in Japan, and yet you lack information, experience, and are unfamiliar with the job-hunting process in most Japanese companies, you will not regret joining the program by ASIA to JAPAN. As joining this doesn’t cost you any yet providing you a platform in contact and matching with lots of companies, also guidance and advice for the interviews. Moreover, the process is fast, where it took around 1-2 weeks from the first interviews until you get the job offers, where it may take months in comparing to the general way in application to the company. Thus, if you are still frustrated in how to get a job in Japan, give it a try and this will help you understand more through the talk with the staff, and might get your desire job for your future career path. Don’t miss it.
１．研究題名：「効率的なハードウェア指向ドロップアウトアルゴリズムの開発」
２．背景：
近年、神経回路又はニューラルネットワークを人工知能に載せて、ロボットや自動運転等の応用に役にたって流行ってます。しかし、ニューラルネットワークの学習が非常に計算コストをかかります、GPUなどの高速ディバイスを使う場合、消費電力が高いです。FPGAやASICなどハードウェアディバイスは低消費電力でリアルタイム処理が可能です。これにより、組込みシステムに広く実装できるようになります。しかし、現在のアルゴリズムは、ほとんどソフトウェアに適し。そのままFPGAに実装したら、多くなリーソスがかかり、並列処理ができません。従って、リーソスの消費を最小限抑え、並列処理を可能にするために、変更されたアルゴリズムが必要です。
ドロップアウトは過剰適合問題を解決するための一般的な手法の一つです。機械学習でよくあり、訓練データに対して学習されているが、未知データに対しては適合できていないことです。ドロップアウトは学習時、ランダムで入力層と中間層のニューロンを落として、過剰適合問題を避けます。しかし、しかし、ニューロンをランダムに落とすことは、ハードウェアで実装することが難しいです。元ドロップアウトは乱数生成器を使って、設定した比率と繰り返す比べます。この手法は並列処理に向けない、多くなリーソスも必要です。
乱数生成器や比較器などリソースかかるのを使わず、並列処理できるのハードウェア指向アルゴリズムを開発します。提案手法では、、ベースとして事前に作ったドロップアウトマスクを用いて、回転またはシャッフルして新しマスクを生成できます。一般的な手法と比べると、同じぐらいの効果を得って（ソフトウェア検証）、リソースの必要が多めに減れるのことを検証しました（ハードウェア検証）。その上で、提案手法が並列処理ができるから、１クロックだけでマスクの生成ができます。
３．研究目的：
提案手法を検証、元の手法と比べ、リソースや正確率を比較のため。
４．過程
１）ブロック図とフローチャートを設計
２）提案手法のpythonコードを書く
３）ニューラルネットワークを学習させる
４）パフォーマンスをを比較
５）Verilogコードを書く
６）消費の合成リソースを比較
５．結果：
１．提案された方法は、オーバーフィット問題を解決できる従来のドロップアウトアルゴリズムと同じ効果を達成しました。 （認識精度の面で比較）
２．並列処理が可能な提案された方法により、単一のクロックサイクルで処理を完了できます（処理速度はクロックの周波数に依存します）
３．リソース消費は3つのアプローチの中で最も少なく、FPGAへの実装で最も効率的です。
６．振り返り
この研究で、ニューラルネットワークに関する知識を学びました。一つアルゴリズムの開発なんですが、全面的に理解しないと、コード書く時、全体のパフォーマンスを影響するのがあります。新しい手法を開発ですから、色々分析や実験が必要です。並列処理がなかなか難しいです。それをよく理解するため、ゆっくりでひとつずつ描く、分析して、近道がないのをわかりました。
My research is to implement hardware oriented neural network algorithm for efficient FPGA implementation. Neural network has outstanding performance in various fields such as automation, robotics and so on. Deep Neural Network can be applied for only inference mode or trainable mode. In inference mode, the DNN uses the pre-trained model and does not update with the changing in time and environment, where in trainable mode, the weights of the model update corresponding to the changes, allow the network to learn. To apply DNN, the processing system can be categorized into three major class, which are software system (CPU, GPU, etc.), hardware system (FPGA, ASIC, etc.) and software-hardware combination system (SoC, etc.)
However, training neural network on device is relatively difficult due to the high computation cost. FPGA is an electronic hardware device that enable real time processing with very low power consumption. Thus it is suitable for application of mobile device and embedded system. However, the implementation of neural network into FPGA is difficult because the difference in architecture resulting common algorithm unable to apply in it efficiently, in term of either resources or performance as well. Moreover, the constraint of resources in FPGA resulting the higher priority in efficiently utilize the resource optimally.
[Problem Statement]
The biggest problem of implement DNN in embedded system is the high power consumption caused by the complex computation. This problem can be tackled by implement with FPGA, which provides real-time processing with low power consumption due to its parallel processing characteristic. However, the current algorithms of neural network are usually developed in CPU or GPU, which is software-based and is not suitable for FPGA. The hardware oriented algorithms are required.
[Solution/Proposed method]
In this research, we focus on dropout algorithm and propose an alternative dropout algorithm which is hardware-oriented, by eliminating the required of random number generator, and enable in parallel processing, resulting real-time processing and resource-saving. The dropout technique is usually applied in training the DNN for avoiding overfitting problem, which is caused by over-train the model and resulting drops in performance.
[Experimental Results]
The experiment had been carried out by separating into two part: software verification and hardware synthesis comparison. In software verification part, various architecture of neural network such as Multi-Layer Perceptron (MLP) and Convolutional Neural Network (CNN) had been trained with different type of dataset (MNIST, CIFAR-10 and etc.) respectively. Experiments were carried out with training the neural networks without dropout algorithm, with conventional dropout, and with the proposed method. Where in the comparison of hardware synthesis, 3 approaches had been carried out to compare the resources consumption and the processing speed (clock cycle). The 3 approaches which are including the conventional dropout in series processing, conventional dropout in parallel processing, and the proposed method.
In this research, several results can be observed and verified through the experiment.
– The proposed method achieved the same effect to the conventional dropout algorithm which can solved the overfitting problem. (Compare in aspect of recognition accuracy)
– The proposed method able to process in parallel, which allow the process complete in a single clock cycle (where the processing speed depend to the frequency of clock)
– The resource consumption is the least compare among the 3 approaches, which is the most efficient in implementation into FPGA.
