First of all you need to create a 2CO account from : https://www.2checkout.com
1. To setup the callback process for 2CheckOut to automatically mark invoices paid, Navigate to Account tab > Site Management from your 2CheckOut account and set "Direct Return" to Yes and the "Approved URL" to http://www.yourdomain.com/whmcs/modules/gateways/callback/2checkout.php (*Replacing http://www.yourdomain.com/whmcs/ with the actual URL of your WHMCS installation)
2. For added security, on the same page you can set a "Secret Word". If set, this is then used to validate callbacks are authentic and it is therefore recommended that you do this. If you choose to enable it, then you must also enter the exact same word that you set inside the 2CheckOut control panel into the WHMCS Secret Word field in Setup > Payments > Payment Gateways.
3. To setup the callback process for 2CheckOut to automatically mark invoices paid, you will need to login to your 2CheckOut account and click the Notifications tab. First click Enable All Notifications. In the Global URL field, enter the url http://www.yourdomain.com/whmcs/modules/gateways/callback/tco.php .. (*Replacing http://www.yourdomain.com/whmcs/ with the actual URL of your WHMCS installation)
4. If you wish to be able to use the recurring billing and/or automatic refunds functionality within WHMCS, then you must setup a 2CheckOut API user and enter the details in these fields. To setup an API user, you will need to login to your 2CheckOut account and go to Account > User Management. Click Create Username. Enter an email address, username & password, security question & answer. Select the permissions for the user. You just need to select API Access & API Updating for the permissions. Enter the same username & password that you just created into the appropriate API username and API password fields inside WHMCS.
Thats it! You have successfully integrate the whmcs with 2checkout now.