When loading the My Learning Modules Web Part, it takes much time to load the web part. This appears to happen for all users. It seems generally that after the web part has loaded successfully once, when you go onto the page again, it is quicker.
MLWP always caches loaded items except the case when it is on the course homepage and retrieving data only from the current course. This may explain why it is loaded faster second time.