Thanks a lot Chris for your detailed response.
I forgot to admit that I got this error when I am trying to install ASE 15.7 on 64 bit operating system only. Total physical memory of the box is 4 GB. So, when this error occurred it just prompted me to with two options next and cancel. So, I moved by clicking next after increasing the virtual memory, then the installation went till the end and I have received the message "Installation failed". However, I am able to access isql from my programs.
Regards,
Dilip Voora