Hi,
This is in C#, but you can easy convert it to VB. I think you can use the Index direcly in VB (eg you dont neet the get_Item, and I also think you dont need the object o).
Code:
// find a dataset, and if exists delete it
object o = "DatasetX";
try {
MapPoint.DataSet DatasetX = MP.ActiveMap.DataSets.get_Item(ref o);
DatasetX.Delete();
} catch {
// Dataset does not exists
}