main| new issue| archive| editorial board| for the authors| publishing house|
Ðóññêèé
Main page
New issue
Archive of articles
Editorial board
For the authors
Publishing house

 

 


ABSTRACTS OF ARTICLES OF THE JOURNAL "INFORMATION TECHNOLOGIES".
No. 3. Vol. 30. 2024

DOI: 10.17587/it.30.115-123

G. S. Veresnikov, Dr. Sc., Leading Researcher, A. V. Skryabin, Researcher,
V. A. Trapeznikov Institute of Control Sciences of RAS, Moscow, 117997, Russian Federation

Artificial Intelligence Methods in Automated Unmanned Aerial Vehicles Control Systems

One of the main problems in ensuring the unmanned aerial systems (UAS) safety and control performance indicators is the operational analysis organization of heterogeneous data coming from on-board sensors and the formation of adequate recommendations and decisions on their basis of flight missions implementation. In recent years, there have been many research papers devoted to solving this problem using artificial intelligence (AI) methods. The article discusses AI methods for using in tasks related to UAS. We have described the sources of information to generate the data necessary for the application of AI methods. We have classified typical tasks of computer vision and navigation systems for solving using AI methods. We have analyzed the generally accepted classification of AI methods within the scope of the research subject. At the same time, special attention is paid to the features of AI methods that allow solving many well-known problems of recognition, approximation, optimization for UAS target and navigation tasks realization and effective operator support. In particular, we have considered neural networks, decision trees, support vector machines, k-nearest neighbors, genetic, ant colony algorithms, artificial immune systems. Currently, the hardware allows integrating complex algorithms based on these methods on board and widely using them in flight missions. The results of the study conducted as part of the review are illustrated by examples from the scientific publications.
Keywords: artificial intelligence methods, unmanned aerial vehicle, computer vision, control, classification, optimization

Acknowledgements: This work has been partially supported by the grants the Russian Science Foundation (project No. 23-19-00464). DOI: 10.17587/it.30.115-123

P. 115-123

References

  1. Taisho T., Enfu L., Kanji T., Naotoshi S. Mining Visual Experience for Fast Cross-view UAV localization, Proceedings of the 8th Annual IEEE/SICE International Symposium on System Integration, 2015, Nagoya, Japan, pp. 375—380.
  2. Lin T.-Y., Cui Y., Belongie S., Hays J. Learning Deep Representations for Ground-to-Aerial Geolocalization, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 5007—5015.
  3. Morito T., Sugiyama O., Kojima R., Nakadai K. Partially Shared Deep Neural Network in Sound Source Separation and Identification Using a UAV-Embedded Microphone Array, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2016, Daejeon, Korea (South), pp. 1299—1304.
  4. Maturana D., Scherer S. 3D Convolutional Neural Networks for Landing Zone Detection from LiDAR, Proceedings of the IEEE International Conference on Robotics and Automation, 2015, pp. 3471—3478.
  5. Zhang T., Kahn G., Levine S., Abbeel P. Learning Deep Control Policies for Autonomous Aerial Vehicles with MPC Guided Policy Search, Proceedings of the IEEE International Conference on Robotics and Automation, 2016, Stockholm, Sweden, pp. 528—535.
  6. Mendis G. J., Randeny T., Wei J., Madanayake A. Deep Learning Based Doppler Radar for Micro UAS Detection and Classification, Proceedings of the IEEE Military Communications Conference (MILCOM), 2016, Baltimore, Md, USA, pp. 924—929.
  7. Bueren S. K., Burkart A., Hueni A., Rascher U. Deploying four optical UAV-based sensors over grassland: Challenges and Limitations, 2015, Biogeosciences, vol. 12, no. 1, pp. 163—175.
  8. Lin S., Garratt M. A., Lambert A. J. Monocular Vision-Based Real-Time Target Recognition and Tracking for Autonomously Landing an UAV in a Cluttered Shipboard Environment, Autonomous Robots, 2017, vol. 41, pp. 881—901.
  9. 9. Yu C., Cai J., Chen Q. Multi-Resolution Visual Fiducial and Assistant Navigation System for Unmanned Aerial Vehicle Landing, Aerospace Science and Technology, 2017, vol. 67, pp. 249—256.
  10. 10. Li H., Duan H. Verification of Monocular and Binocular Pose Estimation Algorithms in Vision-Based UAVs Autonomous Aerial Refueling System, Science China Technological Sciences, 2016, vol. 59, pp. 1730—1738.
  11. 11. Ayalew A. A Review on Object Detection from Unmanned Aerial Vehicle Using CNN, International Journal of Advance Research, Ideas and Innovations in Technology, 2019, vol. 5, iss. 4, pp. 241—243.
  12. 12. Oscoa L. P., Junior J. M., Ramosc A. P. M., Jorgee L. A. , Fatholahi S. N., Silva J. A., Matsubara E. T., Pistori H., Gonyalves W. N., Li J. A Review on Deep Learning in UAV Remote Sensing, International Journal of Applied Earth Observation and Geoinformation, 2021, vol. 102, pp. 102456.
  13. 13. Sultania W., Shah M. Human Action Recognition in Drone Videos Using a Few Aerial Training Examples, Computer Vision and Image Understanding, 2021, vol. 206, pp. 103186.
  14. 14. Huang W., Zhou X., Dong M., Xu H. Multiple Objects Tracking in the UAV System Based on Hierarchical Deep High-Re­solution Network, Multimedia Tools and Applications, 2021, vol. 80, pp. 13911—13929.
  15. 15. Xu Y., Croon G. CNN-based Ego-Motion Estimation for Fast MAV Maneuvers, Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), 2021, Xi'an, China, pp. 7606—7612.
  16. 16. Baldini F., Anandkumar A., Murray R. M. Learning Pose Estimation for UAV Autonomous Navigation and Landing Using Visual-Inertial Sensor Data, Proceedings of the 2020 American Control Conference (ACC), 2020, Denver, CO, USA, pp. 19830747.
  17. 17. Zhang Y., Wang T., Cai Z., Wang Y., You Z. The Use of Optical Flow for UAV Motion Estimation in Indoor Environment, Proceedings of the 2016 IEEE Chinese Guidance, Navigation and Control Conference (CGNCC), 2016, IEEE, Nanjing, pp. 16619062.
  18. Liu X., Li X., Shi Q. UAV attitude estimation based on MARG and optical flow sensors using gated recurrent unit, International Journal of Distributed Sensor Networks, 2021, vol. 17, no. 4.
  19. Zhou B., Lapedriza A., Xiao J., Torralba A., Oliva A. Learning deep features for scene recognition using places database, Proceedings of the 28th Annual Conference on Neural Information Processing Systems, 2014, Montreal, Canada, pp. 487—495.
  20. Wan L., Zhang G. Super-resolution reconstruction of unmanned aerial vehicle image based on deep learning, Proceedings of the 2021 2nd International Conference on Internet of Things, Artificial Intelligence and Mechanical Automation (IoTAIMA 2021), Journal of Physics: Conference Series, 2021, vol. 1948, pp. 012028.
  21. Sadeghi F., Levine S. Real single-image flight without a single real image, Proceedings of the Robotics: Science and Systems, 2017, Cambridge, Massachusetts, USA.
  22. Kelchtermans K., Tuytelaars T. How hard is it to cross the room? — training (recurrent) neural networks to steer a UAV, available at: https://arxiv.org/abs/1702.07600.
  23. Akilandeswari J., Jothi G., Naveenkumar A. et al. Design and development of an indoor navigation system using denoising autoencoder based convolutional neural network for visually im­paired people, Multimedia Tools and Applications, 2022, vol. 81, pp. 3483—3514.
  24. Shah U., Khawad R., Krishna K. M. Deepfly: Towards complete autonomous navigation of mavs with monocular camera, Proceedings of the Tenth Indian Conference on Computer Vision, Graphics and Image Processing, 2016, New York, NY, USA, pp. 59.
  25. Giusti A., Guzzi J., Ciresan D. C. et al. A machine learning approach to visual perception of forest trails for mobile robots, IEEE Robotics and Automation Letters, 2016, vol. 1, no. 2, pp. 661—667.
  26. Pehlivanoglu Y. V., Pehlivanoglu P. An enhanced genetic algorithm for path planning of autonomous UAV in target coverage problems, Applied Soft Computing, 2021, vol. 112, pp. 107796.
  27. Baskaya E., Bronz M., Delahaye D. Fault detection & diagnosis for small UAVs via machine learning, Proceedings of the 36th Digital Avionics Systems Conference (DASC), 2017, IEEE/AIAA, pp. 17354650.
  28. Giagkos A., Tuci E., Wilson M. S., Charlesworth P. B. UAV flight coordination for communication networks: genetic algorithms versus game theory, Soft Computing, 2021, vol. 25, pp. 9483—9503.
  29. Qamar S., Khan S. H., Arshad M. A., Qamar M., Gwak J., Khan A. Autonomous Drone Swarm Navigation and Multitarget Tracking With Island Policy-Based Optimization Framework, IEEE Access, 2022, vol. 10, pp. 91073—91091.
  30. Ashraf A., Majd A., Troubitsyna E. Online Path Generation and Navigation for Swarms of UAVs, Scientific Programming, 2020, vol. 2020, pp. 8530763.
  31. Hung C., Xu Z., Sukkarieh S. Feature learning-based approach for weed classification using high resolution aerial images from a digital camera mounted on a UAV, Remote Sensing, 2014, vol. 6, no. 12, pp. 12037—12054.
  32. Li K., Zhang K., Zhang Z., Liu Z., Hua S., He J. A UAV Maneuver Decision-Making Algorithm for Autonomous Airdrop Based on Deep Reinforcement Learning, Sensors, 2021, vol. 21, iss. 6, pp. 2233.
  33. Bazi Y., Melgani F. Convolutional SVM Networks for Object Detection in UAV Imagery, IEEE Transactions on Geoscience and Remote Sensing, 2018, vol. 56, iss. 6, pp. 3107—3118.
  34. Wei K., Wu J., Ma W., Li H. State of charge prediction for UAVs based on support vector machine, The Journal of Engineering, 2019, vol. 5, pp. 9133—9136.
  35. Bejiga M., Zeggada A., Nouffidj A., Melgani F. A con-volutional neural network approach for assisting avalanche search and rescue operations with UAV imagery, Remote Sensing, 2017, vol. 9, no. 2, pp. 100.
  36. Wang L., Misra G., Bai X. A K-Nearest Neighborhood-Based Wind Estimation for Rotary-Wing VTOL UAVs, Drones, 2019, vol. 3, iss. 2, pp. 31.
  37. Fernando L., Ruiz C., Guasselli L. A., Caten A., Za-notta D. C. Iterative K — Nearest Neighbors Algorithm (IKNN) for submeter spatial resolution image classification obtained by Unmanned Aerial Vehicle (UAV), International Journal of Remote Sensing, 2018, vol. 39, iss. 15—16, pp. 5043—5058.
  38. Chen C.-W., Hsieh P.-H., Lai W.-H. Application of deci­sion tree on collision avoidance system design and verification for quadcopter, Proceedings of the International Conference on Unmanned Aerial Vehicles in Geomatics, 2017, vol. XLII-2/W6, pp. 71—75.
  39. Smith J. F., Vu T., Nguyen H. Fuzzy decision trees for planning and autonomous control of a coordinated team of UAVs, Proceedings of the International Society for Optical Engineering, 2007, vol. 6567, pp. 656708.
  40. Feng Q., Liu J., Gong J. UAV Remote Sensing for Urban Vegetation Mapping Using Random Forest and Texture Analysis, Remote Sensing, 2015, vol. 7, iss. 1, pp. 1074—1094.
  41. Moradi F., Samadzadegan F., Javan F. D., Toosi A. Tree detection using UAV-based imagery based on Random Forest clas­sification, Proceedings of The 2nd International Electronic Confe­rence on Forests — Sustainable, 2021.
  42. McLaughlin R. G. Artificial Immune System for Unmanned Aerial Vehicle Abnormal Condition Detection and Identification, Graduate Theses, Dissertations, and Problem Reports, 2019, pp. 7413.
  43. Kaneshige J., Krishnakumar K. Artificial immune system approach for air combat maneuvering, Proceedings of the SPIE — The International Society for Optical Engineering, 2017, vol. 6560, pp. 656009.
  44. Bhandari S., Raheja A., Tang D., Ortega K., Dadian O., Bettadapura A. Nonlinear control of UAVs using multi-layer perceptrons with off-line and on-line learning, Proceedings of the American Control Conference, 2014, Portland, OR, USA, pp. 2875—2880.
  45. Xiao-Wei W. A Multilayer Perceptron Neural Network Model for UAV Sensor Fault Detection, Proceedings of the International Conference on Information Systems and Computer Aided Education, 2021, Dalian, China, pp. 21463862.
  46. Dadian O., Bhandari S., Raheja A. A recurrent neural network for nonlinear control of a fixed-wing UAV, Proceedings of the American Control Conference, 2016, Boston, MA, USA, pp. 16193824.
  47. Sarma K. K., Mitra A. A Deep Learning Recovery Mechanism using LSTM for UAV Channel Modeling, Journal of The Institution of Engineers (India) Series B, 2021, vol. 107, pp. 550—557.
  48. Gayathri D., Sowmiya N., Yasoda K., Muthulakshmi K., Kishore B. Review on application of drones for crop health monitoring and spraying pesticides and fertilizer, Journal of critical reviews, 2020, vol. 7, pp. 667—672.
  49. Girisha S., Ujjwal V., Manohara M., Radhika P. UVid-Net: Enhanced Semantic Segmentation of UAV Aerial Videos by Embedding Temporal Information, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, vol. 99, pp. 4115—4127.
  50. Beeharry Y., Bassoo V. Performance of ANN and AlexNet for weed detection using UAV-based images, Proceedings of the 3rd International Conference on Emerging Trends in Electrical, Electronic and Communications Engineering (ELECOM), 2020, pp. 163—167.
  51. 51. Levkovits-Scherer D. S., Cruz-Vega I., Martinez-Carranza J. Real-Time Monocular Vision-Based UAV Obstacle Detection and Collision Avoidance in GPS-Denied Outdoor Environments Using CNN MobileNet-SSD, Proceedings of the Mexican International Conference on Artificial Intelligence: Advances in Soft Computing, Lecture Notes in Computer Science, 2019, vol. 11835, pp. 613—621.
  52. Choudhari V., Phadtare M., Pedram R., Vartak S. Comparison between YOLO and SSD MobileNet for Object Detection in a Surveillance Drone, International journal of scientific research in engineering and management, 2021, vol. 05, iss. 10.
  53. Levkovits-Scherer D. S., Cruz-Vega I., Martinez-Carran-za J. Real-Time Monocular Vision-Based UAV Obstacle Detection and Collision Avoidance in GPS-Denied Outdoor Environments Using CNN MobileNet-SSD, Proceedings of the Mexican International Conference on Artificial Intelligence: Advances in Soft Computing, Lecture Notes in Computer Science, 2019, vol. 11835, pp. 613—621.
  54. Sonmez A., Kocyigit E., Kugu E. Optimal path planning for UAVs using Genetic Algorithm, Proceedings of the International Conference on Unmanned Aircraft Systems (ICUAS), 2015, Colorado, USA, pp. 50—55.
  55. Macharet D. G., Neto A. A., Campos M. F. M. Feasible UAV Path Planning Using Genetic Algorithms and Bazier Curves, Proceedings of the SBIA 2010: Advances in Artificial Intelligence, 2010, pp. 223—232.
  56. Konatowski S., Pawlowski P. Ant colony optimization algorithm for UAV path planning, Proceedings of the 14th International Conference on Advanced Trends in Radioelecrtronics, Telecommunications and Computer Engineering (TCSET), 2018, Lviv-Slavske, Ukraine, pp. 17686175.
  57. Muntasha G., Karna N., Shin S. Y. Performance Analysis on Artificial Bee Colony Algorithm for Path Planning and Collision Avoidance in Swarm Unmanned Aerial Vehicle, Proceedings of the International Conference on Artificial Intelligence and Mechatronics Systems (AIMS), 2021, IEEE, pp. 20895999.
  58. Li P., Duan H. Path planning of unmanned aerial vehicle based on improved gravitational search algorithm, Science China Technological Sciences, 2012, vol. 5, iss. 10, pp. 2712—2719.
  59. Xie C., Zheng H. Application of Improved Cuckoo Search Algorithm to Path Planning Unmanned Aerial Vehicle, Proceedings of the Intelligent Computing Theories and Application, 2016, pp. 722.729.
  60. Kim I., Matos-Carvalho J. P., Viksnin I., Simas T., Correia S. D. Particle Swarm Optimization Embedded in UAV as a Method of Territory-Monitoring Efficiency Improvement, Symmetry, 2022, vol. 14, no. 6, pp. 1080.

To the contents