Hi
Please Go to,
Wp-admin > Appearance > Editor > Select Function.php file
and make the change as I did in the file below and Save
Refer: http://prntscr.com/mahsce
If you still have any issue
Please share your website URL with working website credentials in private reply.
Well you can enable : " Open Login popup for non logged in users when they click on take this course button(For all courses) " from wp-admin -> lms-> settings .
TO show registration form there please enable ajax login and registration from wp-admin -> WPLMS -> buddypress .
and create a custom registration : https://wplms.io/support/knowledge-base/custom-registration-forms-in-wplms/