Tuesday, May 27, 2014

Overview of Database Normalization

Normalization reduces data redundancy and inconsistent data dependency to avoid all anamolites(Deletion,Insertion and updation)

1NF: No Repeiting Column value
2NF: No Partial functional dependency
3NF: No Transitive dependency

A database table is said to be in 1NF if it contains no repeating fields/columns
Separate the repeating fields into new database tables along with the key from unnormalized database table.
The primary key of new database tables may be a composite key
A database table is said to be in 2NF if it is in 1NF and contains only those fields/columns that are functionally dependent(means the value of field is determined by the value of another field(s)) on the primary key
Remove the partial dependencies(A type of functional dependency where a field is only functionally dependent on the part of primary key) of any non-key field.
If field B depends on field A and vice versa. Also for a given value of B, we have only one possible value of A and vice versa, Then we put the field B in to new database table where B will be primary key and also marked as foreign key in parent table.
A database table is said to be in 3NF if it is in 2NF and all non keys fields should be dependent on primary key(2NF) and no fields of the table is transitively functionally dependent on the primary key.
    Remove the transitive dependecies(A type of functional dependency where a field is functionally dependent on the Field that is not the primary key.Hence its value is determined, indirectly by the primary key )
Make separate table for transitive dependent Field.


KEY that can uniquely determine records

https://www.youtube.com/watch?v=G9SA0Yv-o28
https://www.simple-talk.com/community/forums/thread/111401.aspx
http://holowczak.com/database-normalization/


Thursday, January 30, 2014

By the Morning brightness

morning
by Raiiq Ridwan
Tough times come in the life of every human being, young or old, rich or poor, king or slave. And tough times, more often than not, make us who we are.
In fact, tough times (and learning to deal with them) help our true nature emerge. For example, during bad times a really “nice” person might show complete impatience and displeasure at the will of Allah (swt), whereas a not-so-nice person might actually turn towards Allah in times of need, bringing about a change in his life that puts him among the pious.
The Way Out of Depression
In recent times depression and sadness have been rife in the Earth. We have more than ever before but we’re more depressed than ever before. Anti-depressants are fast becoming the most prescribed drug, suicide is high, and morality has gone to an all-time low. The Messenger used to say, The best speech is the speech of Allah, and the best of guidance is the guidance of Muhammad.
Surah Duha
A man who is depressed, sad, or grieving requires the nurturing and caring of a well-wishing soul more than he needs the pill that alleviates his symptoms for a little while. Who better to speak to you to calm you down than Allah (swt)? And which better way but the way of the messenger? No human being suffered more than him, and it was at one of the saddest points of Mohamed’s life that Allah (swt) revealed Surah Duha.
The Messenger has just started giving the message to his people who were mocking and making fun of him. At that time revelation stopped for six months and their mockery and slandering just increased as they said things like, “No revelation today? Your Lord surely hates you now!”
Wake Up and See the Sunshine
So often do we not feel that Allah (swt) really hates us? He must hate us for all the pain we get, right? No— the answer comes!
By the morning brightness! [Quran, 93:1]
Your life was all hustle and bustle like the morning brightness! It was bright and sunny before, and don’t worry, it still is today! Wake up, look at the sunshine, look at life around you— it’s not all doom and gloom.
And by the night when it grows still . . . [Quran, 93:2]
Yes, a depressed life is like the stillness of the night; nothing moves, nothing seems to be going forward. But what comes after the stillness of the night? The bright morning! It was a bright morning only hours earlier, and another bright morning is coming! Its okay, don’t worry, good times are ahead. Get some sleep.
Allah (swt) Loves You and Has not Deserted You
Your Master has not said goodbye to you nor does He hate! [Quran, 93:3]
Your Master has not even said goodbye, so don’t worry! Wadda’a in Arabic is a loving goodbye. Your Master has not even lovingly said goodbye to you and doesn’t “hate.” Still not impressed? Look at how Allah (swt) gives you evidence that He doesn’t hate you or that He hasn’t said goodbye to you.
Things Will Get Better!
And the future will be better for you than the past. [Quran, 93:4]
Look forward. After bad times come good, and after pain comes ease. And if not, then the believer knows that after death comes paradise. Yes, it’s painful, but hold on, the future will be better!
Your Master will give you so much that you will be pleased. [Quran, 93:5]
Don’t worry, be patient. A time is coming that Allah will give you so much that you will be pleased. For many of us in this world, if not then at least in the hereafter in sha Allah! Imagine all that you wanted. Your wildest dreams that you know will never come true, Allah will make them all come true in paradise. You still need proof right?
Remember What He’s Already Saved You From
Did he not find you an orphan and shelter you? [Quran, 93:6]
Many of you were orphaned once upon a time, either you lost your father or both father and mother. But didn’t Allah help you out of it? Aren’t you better now? What about us with both parents? Were we not helpless and small babies, and didn’t Allah give us parents as a shelter? Or maybe uncles and aunties? Or brothers and sisters? Did Allah not give us a shelter?
And He found you lost(or searching) and guided you. [Quran, 93:7]
Were we not lost and wandering in the ocean of misguidance once and Allah guided us? Were we not searching for happiness everywhere (some of us still might be) whereas it was in the guidance of Allah? Ask Allah (swt) to guide you through man. Look what He did for you before, is He really going to forsake and bid you goodbye?
Remember His Generousity to You
And He found you in need and satisfied your need? [Quran, 93:8]
How many a times have we needed something and it was satisfied by Allah (swt)? We needed parents to take care of us and Allah (swt) gave that to us. We needed food and Allah created the world in such a way that we can extract food. We needed help and Allah (swt)created someone to help us. We needed and He satisfied? Are you yet not convinced that Allah (swt) is going to help? If you are, now time to take action. What do you do?
So do not be harsh with the orphan. And do not turn away the one who asks for help. [Quran, 93: 9]
In short, get active in the community. Help those who are in a worse condition than you are. That will humble your heart, take your mind off your own problems as you focus on others’. It will make you a stronger man and you will also be reminded of the bounties of God in your life and you will thus be grateful. Did I say grateful?
Talk About the Blessings of Your Master
Any of those people writing books on “being happy” or on “positive thinking” will tell you to look at the good side of life. This is exactly what Allah (swt) prescribes, just that He did 1400 years before today in the Quran and also in previous scriptures. Talk about the blessings Allah (swt) has given you. You’ve whined and complained enough. Now talk about how much He’s given you and thank Him for it. When was the last time we thanked Him for our eyes? For good skin? For not having a stomach ulcer? For the immune system?
Talk about the blessings of your Master— therein is peace.

The Psychology of Shukar(Gratitude)


by Bela Khan
She opens her wardrobe. Two dozen dresses— red, yellow, orange, pink, and you-name-it— are hanging there.  After a cursory glance, she bangs the door shut.
“Mom, I have nothing to wear for the party,” she whines.  “I’ve been begging you to take me shopping for ages— you never listen to me!  What do I do now?”
He’s rummaging in the fridge for the fourth time.  Pizza, chocolate, ice-cream, fruits, and juices are stuffed inside.  He closes the door despondently and leaves the kitchen, saying, “There’s nothing much to eat in here. Let’s go to the cafĂ© and get some decent food.”
She wakes up with the same feelings of depression and dread that she’s been experiencing every morning.  Thoughts of suicide are again floating in her mind.  She’s married to a handsome man and has two lovely kids, a home, and a car— everything most women would ask for.  Apparently, life is perfect, but something is missing; she’s just not married to the man of her dreams.
He’s seventeen years old. He has an iPhone, Core I-7 Dell, and 1000 bucks monthly pocket money, but according to him his life sucks.  He can’t focus on his textbook.  He slams it shut and starts pacing the room as he thinks aloud: “If my friend can own a Jaguar, why the hell can’t I? Everything sucks.”
Do any of these scenarios sound familiar?
 The Ingratitude Habit
As we grow up, we get tuned to be ungrateful, to be blind to the blessings in our lives. We focus all our energy and attention on looking for what’s lacking in our lives. This is really ironic, because the more we focus on scarcity, the more we tend to attract it.
How to Become a Black Hole of Despair
Research has proven that our thoughts and emotions emit powerful signals, and that these signals have the ability to attract signals of the same frequency!  In other words, whenever you’re ungrateful or have negative, self-defeating thoughts (for example, I never seem to have enough of money, or I can never be healthy, or I’ll never find a practicing husband) or worry about issues beyond your control, you’re actually emitting energies that will attract similar energies. Ultimately, you’ll end up becoming a black hole of despair.  All the worries, arguments, and frustration just lead to more of the same.
Want to try to attract more peace, joy, harmony, ease, and prosperity? Choose your thoughts consciously and carefully.
Open Secrets in the Quran
Today non-Muslim researchers have discovered these secrets in the name of natural laws, while these “open secrets” were given to us in the Quran 1400 years ago. Psychologists and philosophers have painstakingly conducted research and come up with new theories and ideas.  All this “newly discovered” stuff was presented to us in the Quran in no uncertain terms.  Allah (swt) promises us: If you are grateful, I’ll give you more.
And [remember] when your Lord proclaimed, ‘If you are grateful, I will surely increase you [in favor]; but if you deny, indeed, My punishment is severe.’[Quran, 14:7]

This is a ready-made formula for attracting more of what you want in life.
Surah Fatihah: A Pillar’s Pillar
It is well-known that prayer, the important pillar of Islam, is incomplete without Surah Fatihah.  Surah Fatihah begins with expressing our gratitude to our Lord for the countless blessings He has showered upon us.
Have you ever noticed that a lot of duas taught by the Prophet (peace and blessings be upon him) also start with the same expression of gratitude—alhamdulillah? The day starts with the supplication Alhmdulillahilazi ahyana… (All praise and thanks be to the one who gave us life.)
Allah (swt) wants you to start your day in a super-charged state, attracting ample happiness and abundance in life. The meal ends with the supplicationAlhmdulillahizli at’amana wa sakana . . . (All  praise and thanks be to the one who gave us food and drink). Why? Because He always wants you to have better food every time you eat.  The same expression— alhamdulillah; it’s not a coincidence, is it?
Getting Back on Track
Gratitude is one of the quickest ways to regain our focus. When we feel stuck it means we’re giving too much attention to something that doesn’t really deserve it. Most so-called problems are usually mundane preoccupations and mental obsessions. We focus so much on one area to the exclusion of everything else that we create and thus magnify our problems.
Surprisingly, when we focus so much on what’s not working, we can’t even access the part of our brain that provides us with solutions. So, when something is not working, we should focus on what is working and sayalhmdulillahi rabbil 3lameen aloud.
What do you think you’re doing by saying this?  You’re freeing your mind from those thoughts that bog you down and reinforcing strong, positive beliefs that even if some areas are not working in life, others definitely are. In this way, you’re able to think in a quicker and better way to resolve what’s not working in life. Try it out!
It’s reported that some of the companions of the Holy Prophet (pbuh) were even grateful for the calamities they faced and presented their gratitude to Allah (swt) for not letting something worse befall them.  Now that’s the attitude of gratitude!
Being grateful to Allah (swt) does wonders. When you’ll acknowledge His hand in your life, miracles will happen. You’ll be in the middle of a problem, not knowing what might happen next, and suddenly the solution will dawn upon you. You might be in dire need of money and someone might, for example, return to you an old loan that you completely forgot about. Or you might be missing your loved ones and they give you a call right then.
We usually ignore many things that are readily available to us and need a big reason to be grateful. Our family, friends, body, health, job, leisure time, education, food, wisdom, smiles, children, and even the painful lessons that life teaches us (and the list goes on) are huge blessings to be grateful for. If you don’t have the same feeling, just close your eyes and imagine one of these blessings vanishing from your life. How does that feel? This is exactly the reason you need to be thankful each day, every moment.
All of us need a bit of fine tuning, so below are some tools that I am sharing with you to enable you to be grateful to Him and let the magic unfold.
Essential Tools for a Grateful Life:
1). Make a list of twenty things you appreciate right now. Write alhamdullilahin front of every blessing. Make a list of the things you feel good about and why you’re grateful for that thing, situation, or person in your life. How did you feel doing this? Here’s the good news: you just increased your positive emissions and now you are actually able to attract more happiness, abundance, and prosperity in your life.
2). Start looking for everything you can be grateful for in life and start talking about it all the time no matter how insignificant it is. If you find a buck from your old shirt’s pocket, celebrate. Go around and tell everyone how grateful you are for that. That’s how you are actually tuning your mind to show gratitude for tiniest blessing in your life.
3). At least once a day, share a gratitude status on Facebook and Twitter (or whatever social media you use) and ask others what they are grateful for. Insha Allah, you’ll get ajar for your shukr and theirs too.
4).  Try to seek the positive in every negative. Take out at least three good lessons from every painful experience and be grateful that life threw that situation at you so that you could learn.
Do share with us a list of some blessings in your life in the comments below.
Bela Khan is a student of Islam, a Muslim women empowerment coach, speaker, writer, and philanthropist. Her mission is, as she says, “to make every single woman reconnect with her Lord, refine her relationships, and get the money flow by inculcating an entrepreneurial mindset.”

Wednesday, January 29, 2014

Two Keys to Success Here and in the Hereafter

Two Keys to Success Here and in the Hereafter


By Sumbal Tariq
s there anyone who says that he or she doesn’t want success? Definitely not. Everyone wants to be successful in this life and also in the hereafter. If you put the simple ingredients of success into your life, you get peace of heart and stability of heart mind and soul. These two keys are:
Tawakal (trust) and Sabr (patience)

Out of His infinite love, Allah (swt) always wants matters to be smooth and easy for us. That’s why in many Quranic verses He guides us to develop patience and to have tawakal (trust) in Him.

Trust (Tawakal) and Its Benefits
Tawakal is a state in which we have complete reliance and total dependence on Allah (swt). It means that we must believe and realize that everything we do and everything that happens to us comes from Allah (swt) and that He is in charge of everything. Allah is telling us that to be successful we must develop trust in Him.

Say: “Nothing will happen to us except what Allah has decreed for us: He is our protector. In Allah let the believers put their trust.” [Quran, 9:51]

Say: “He is (Allah) Most Gracious: We have believed in Him, and in Him we have put our trust. So, soon you will know which of us it is that is in manifest error.” [Quran, 67:29]

Once you develop this belief, you won’t keep getting tense in hard times and you’ll understand this reality that all hardships are from Allah and that the purpose of trials is just to test your belief and to develop your complete reliance on Him. In fact Allah (swt) wants to test to find who has reliance on Him. And those who have complete reliance on Him are rewarded. What greater success than to please the king of kings?

Tawakal is not only necessary in difficult times but it’s equally important in good times. In pleasurable moments, realizing the fact that all blessings and happiness of heart are provided by Allah (swt) helps you to understand your own nothingness in front of His greatness.

Patience (Sabr) and Its Benefits
The first question that comes to mind is: why do we need patience? We need it when Allah (swt) takes away or withholds any blessings that we love, such as our jobs, status, wealth, children, belongings, etc. At that time you need to realize that it is Allah who has control over everything. Internalizing this reality helps us to withstand any loss, to scoop out hurt from your heart and to bring calm.

People usually find it difficult to have patience in bad times, but when we hold to the belief that Allah (swt) is the true and strong handhold and that He is Al-wakeel, the best disposer of all affairs, we can easily hand over our matters to Him.

Put your trust in Allah, and Allah is All-sufficient as a Disposer of affairs. [Quran, 33:3]

One of the best examples of patience is Hazrat Yaqoob (as). Allah (swt) took away all his belongings including his children, wealth and even health, but his heart was in rest because he has tawakal on Allah and he was patient. Knowing his condition of heart, Allah (swt) rewarded him with much more and he got eternal success. Allah (swt) love those who are patient, so who is more successful than a person who is loved by Allah?

“Patience is made a condition of success and prosperity. O ye who believe! Persevere in patience and constancy; vie in such perseverance; strengthen each other; and fear Allah; that you may prosper”.  [Quran, 3:200]

Say: “O you My servants who believe!  Fear your Lord. Good is (the reward) for those who do good in this world. Spacious is Allah’s earth! Those who patiently persevere will truly receive a reward without measure!” [Quran,39:10]

Apply these believes in your every single life matters and you will observe settlement of your affairs. Once you rely on Allah (swt) not your own means and withstand difficult situations with patience then your heart get rid of pride, arrogance, self-reliance and ego from the heart. You feel strong in handling your affairs in work, school office, home and where ever you go. This gives you success in this life and also hereafter. Learn it and apply it.

Thursday, August 9, 2012

Vidoes

Why need to follow only one region… why cant we take all religion good things and lead good life?


Asking question to Athiest



Will non muslim go to heaven?



Thursday, June 14, 2012

Speed Reading

http://www.fourhourworkweek.com/blog/2009/07/30/speed-reading-and-accelerated-learning/

How much more could you get done if you completed all of your required reading in 1/3 or 1/5 the time?
Increasing reading speed is a process of controlling fine motor movement—period.
This post is a condensed overview of principles I taught to undergraduates at Princeton University in 1998 at a seminar called the “PX Project”. The below was written several years ago, so it’s worded like Ivy-Leaguer pompous-ass prose, but the results are substantial. In fact, while on an airplane in China two weeks ago, I helped Glenn McElhose increase his reading speed 34% in less than 5 minutes.
I have never seen the method fail. Here’s how it works…

The PX Project

The PX Project, a single 3-hour cognitive experiment, produced an average increase in reading speed of 386%.
It was tested with speakers of five languages, and even dyslexics were conditioned to read technical material at more than 3,000 words-per-minute (wpm), or 10 pages per minute. One page every 6 seconds. By comparison, the average reading speed in the US is 200-300 wpm (1/2 to 1 page per minute), with the top 1% of the population reading over 400 wpm…
If you understand several basic principles of the human visual system, you can eliminate inefficiencies and increase speed while improving retention.
To perform the exercises in this post and see the results, you will need: a book of 200+ pages that can lay flat when open, a pen, and a timer (a stop watch with alarm or kitchen timer is ideal). You should complete the 20 minutes of exercises in one session.
First, several definitions and distinctions specific to the reading process:
A) Synopsis: You must minimize the number and duration of fixations per line to increase speed.
You do not read in a straight line, but rather in a sequence of saccadic movements (jumps). Each of these saccades ends with a fixation, or a temporary snapshot of the text within you focus area (approx. the size of a quarter at 8? from reading surface). Each fixation will last ¼ to ½ seconds in the untrained subject. To demonstrate this, close one eye, place a fingertip on top of that eyelid, and then slowly scan a straight horizontal line with your other eye-you will feel distinct and separate movements and periods of fixation.
B) Synopsis: You must eliminate regression and back-skipping to increase speed.
The untrained subject engages in regression (conscious rereading) and back-skipping (subconscious rereading via misplacement of fixation) for up to 30% of total reading time.

C) Synopsis: You must use conditioning drills to increase horizontal peripheral vision span and the number of words registered per fixation.
Untrained subjects use central focus but not horizontal peripheral vision span during reading, foregoing up to 50% of their words per fixation (the number of words that can be perceived and “read” in each fixation).

The Protocol

You will 1) learn technique, 2) learn to apply techniques with speed through conditioning, then 3) learn to test yourself with reading for comprehension.
These are separate, and your adaptation to the sequencing depends on keeping them separate. Do not worry about comprehension if you are learning to apply a motor skill with speed, for example. The adaptive sequence is: technique ‘ technique with speed ‘ comprehensive reading testing.
As a general rule, you will need to practice technique at 3x the speed of your ultimate target reading speed. Thus, if you currently read at 300 wpm and your target reading speed is 900 wpm, you will need to practice technique at 1,800 words-per-minute, or 6 pages per minute (10 seconds per page).
We will cover two main techniques in this introduction:
1) Trackers and Pacers (to address A and B above)
2) Perceptual Expansion (to address C)

First – Determining Baseline

To determine your current reading speed, take your practice book (which should lay flat when open on a table) and count the number of words in 5 lines. Divide this number of words by 5, and you have your average number of words-per-line.
Example: 62 words/5 lines = 12.4, which you round to 12 words-per-line
Next, count the number of text lines on 5 pages and divide by 5 to arrive at the average number of lines per page. Multiply this by average number of words-per-line, and you have your average number of words per page.
Example: 154 lines/5 pages = 30.8, rounded to 31 lines per page x 12 words-per-line = 372 words per page
Mark your first line and read with a timer for 1 minute exactly-do not read faster than normal, and read for comprehension. After exactly one minute, multiply the number of lines by your average words-per-line to determine your current words-per-minute (wpm) rate.

Second – Trackers and Pacers

Regression, back-skipping, and the duration of fixations can be minimized by using a tracker and pacer. To illustrate the importance of a tracker-did you use a pen or finger when counting the number of words or lines in above baseline calculations? If you did, it was for the purpose of tracking-using a visual aid to guide fixation efficiency and accuracy. Nowhere is this more relevant than in conditioning reading speed by eliminating such inefficiencies.
For the purposes of this article, we will use a pen. Holding the pen in your dominant hand, you will underline each line (with the cap on), keeping your eye fixation above the tip of the pen. This will not only serve as a tracker, but it will also serve as a pacer for maintaining consistent speed and decreasing fixation duration. You may hold it as you would when writing, but it is recommended that you hold it under your hand, flat against the page.
1) Technique (2 minutes):
Practice using the pen as a tracker and pacer. Underline each line, focusing above the tip of the pen. DO NOT CONCERN YOURSELF WITH COMPREHENSION. Keep each line to a maximum of 1 second, and increase the speed with each subsequent page. Read, but under no circumstances should you take longer than 1 second per line.
2) Speed (3 minutes):
Repeat the technique, keeping each line to no more than ½ second (2 lines for a single “one-one-thousand”). Some will comprehend nothing, which is to be expected. Maintain speed and technique-you are conditioning your perceptual reflexes, and this is a speed exercise designed to facilitate adaptations in your system. Do not decrease speed. ½ second per line for 3 minutes; focus above the pen and concentrate on technique with speed. Focus on the exercise, and do not daydream.

Third – Perceptual Expansion

If you focus on the center of your computer screen (focus relating to the focal area of the fovea in within the eye), you can still perceive and register the sides of the screen. Training peripheral vision to register more effectively can increase reading speed over 300%. Untrained readers use up to ½ of their peripheral field on margins by moving from 1st word to last, spending 25-50% of their time “reading” margins with no content.
To illustrate, let us take the hypothetical one line: “Once upon a time, students enjoyed reading four hours a day.” If you were able to begin your reading at “time” and finish the line at “four”, you would eliminate 6 of 11 words, more than doubling your reading speed. This concept is easy to implement and combine with the tracking and pacing you’ve already practiced.
1) Technique (1 minute):
Use the pen to track and pace at a consistent speed of one line per second. Begin 1 word in from the first word of each line, and end 1 word in from the last word.
DO NOT CONCERN YOURSELF WITH COMPREHENSION. Keep each line to a maximum of 1 second, and increase the speed with each subsequent page. Read, but under no circumstances should you take longer than 1 second per line.
2) Technique (1 minute):

Use the pen to track and pace at a consistent speed of one line per second. Begin 2 words in from the first word of each line, and end 2 words in from the last word.
3) Speed (3 minutes):
Begin at least 3 words in from the first word of each line, and end 3 words in from the last word. Repeat the technique, keeping each line to no more than ½ second (2 lines for a single “one-one-thousand”).
Some will comprehend nothing, which is to be expected. Maintain speed and technique-you are conditioning your perceptual reflexes, and this is a speed exercise designed to facilitate adaptations in your system. Do not decrease speed. ½ second per line for 3 minutes; focus above the pen and concentrate on technique with speed. Focus on the exercise, and do not daydream.

Fourth – Calculate New WPM Reading Speed

Mark your first line and read with a timer for 1 minute exactly- Read at your fastest comprehension rate. Multiply the number of lines by your previously determined average words-per-line to get determine your new words-per-minute (wpm) rate.
Congratulations on completing your cursory overview of some of the techniques that can be used to accelerate human cognition (defined as the processing and use of information).
Final recommendations: If used for study, it is recommended that you not read 3 assignments in the time it would take you to read one, but rather, read the same assignment 3 times for exposure and recall improvement, depending on relevancy to testing.
Happy trails, page blazers.

Clean up spaces in DS

Cleaning up the spaces in DataStage


Clearing Lookup Table files
Problem
When a DataStage job with a lookup stage aborts, there may be lookuptable files left in the resource directories and they will consume space. The filenames are similar to "lookuptable.20091210.513biba"
Cause
When a job aborts it leaves the temporary files for postmortem review in the resource directories. Usually that is done in scratch, however, for lookup files, they are created in resource. Lookup filesets will not go away, just like regular datasets.
A lookup fileset looks like:
/opt/IBM/InformationServer/Server/Datasets/export.dsadm.abcdefg.P000000_F0000

A lookup file looks like:
/opt/IBM/InformationServer/Server/Datasets/lookuptable.20091210.513biba
Diagnosing the problem
Look for files with filenames similar to "lookuptable.yyyymmdd.nnnnnnn" left on disk when no jobs are running.
Resolving the problem
All files with lookuptable at the beginning of the filename can be removed as long as there are no running jobs. These files get recreated with every new run of the job and are never reused. If the job runs successfully, then only the lookuptable file created during that job run is removed.
Link
command
ls -ltr | grep 'lookuptable*'
ls -ltr | grep 'export*'
rm –rf <files> (If anything returns from above command and any job is not running)

Clearning &PH& files
Cause
There is a &PH& directory in each DataStage project directory. Files in the &PH& directories under DataStage project directories store runtime information when jobs are running and need to be cleared out periodically.
Answer
To clear the &PH& directory from within DataStage:
1.       Ensure there are no DataStage jobs running anywhere on the system by running "ps -ef | grep dsrpc"
2.       From the DataStage Administrator, go to the Projects page, select the project whose file you want to clear and click the Command button. The Command Interface dialog box opens.
3.       Type the following into the command field: CLEAR.FILE &PH& (all uppercase)
4.       Click Execute to run the command and clear the file.


You can also remove the contents of the &PH& directory using the rm command. However, we suggest that this is scheduled for when the system is least used.

** Important **
Please only delete the contents of &PH& directory and not the &PH& directory itself.

You will find a directory for the &PH& in each project that you have created on that server.

You must not remove any contents or directories for the following:
DS_TEMP*
RT_BP*.O
RT_BP*
RT_LOG*
RT_STATUS*
RT_CONFIG*

These directories/files are related to the jobs within your projects that you are running on the server and if you manually remove or edit any of these files you are likely to corrupt the jobs and possibly the projects. These directories exist for each job that has been created within your project. You could ask your developers to review if there are any jobs that are no longer used and can be removed using the DataStage clients. You can additionally clear down some of the RT_LOG* files by clearing down any
large log files that exist for some of the jobs. This can be done from within DataStage Director client:
1.       Select the job
2.       Click on Job > Clear Log > Immediate purge (Clear all Entries).
You can create a shell script to manually delete the files. To ensure there are no locks only delete files that are from finished jobs. You need to make sure the files are older then the longest running job. Generally you can just delete files older then a week.
                DSPROJDIR=/opt/IBM/Ascential/DataStage/Projects
for project in `ls -l ${DSPROJDIR} | grep "^d" | grep -v "lost+found" | awk '{print $9}'`
do
find "/opt/IBM/Ascential/DataStage/Projects/$project/&PH&"  -mtime +13 -exec rm -f {} \;
done
                Link
                                http://www-304.ibm.com/support/docview.wss?uid=swg21457983

Clearning DataSet files
Data sets can be managed using the Data Set Management tool, invoked from the Tools > Data Set Management menu option within DataStage Designer (DataStage Manager in the 7.5 releases.) Alternatively, the 'orchadmin' command line program can be used to perform the same tasks.
The files which store the actual data persist in the locations identified as resource disks in the configuration files. These files are named according to the pattern below:

descriptor.user.host.ssss.pppp.nnnn.pid.time.index.random

descriptor: Name of the data set descriptor file.
user: Your user name.
host: Hostname from which you invoked the job which created the data set.
ssss: 4-digit segment identifier (0000-9999)
pppp: 4-digit partition identifier (0000-9999)
nnnn: 4-digit file identifier (0000-9999) within the partition
pid: Process ID of the job on the host from which you invoked the jop that creates the data set.
time: 8-digit hexadecimal time stamp in seconds.
index: 4-digit number incremented for each file.
random: 8 hexadecimal digits containing a random number to insure unique file names.


For example, suppose that your configuration file contains the following node definitions:

{
    node node0
    {
         fastname "host1"
         pools ""
         resource disk "/orch/s0" {pools ""}
         resource scratchdisk "/scratch" {pools ""}
    }
    node node1
    {
         fastname "host1"
         pools ""
         resource disk "/orch/s0" {pools ""}
         resource scratchdisk "/scratch" {pools ""}
    }
}

A data set named mydata.ds created by a job using this configuration file will contain data in two partitions, one for each processing node declared in the configuration file. Because each processing node contains only a single disk specification, each partition of data would be stored in a single file on each processing node. Following the naming convention shown above, the data file for partition 0 would be located on the host1 machine, in the /orch/s0 filesystem, and the file would be named:

/orch/s0/mydata.ds.user1.host1.0000.0000.0000.1fa98.b61345a4.0000.88dc5aef

The data file for partition 1 data would be similarly named:

/orch/s0/mydata.ds.user1.host1.0000.0001.0000.1fa98.b61345a4.0001.8b3cb144

It is important to understand that the file referenced in the job, called mydata.ds in our example, does not contain any actual data. It is a data set descriptor file, and it contains information about how the data set is constructed. In order for DataStage jobs to access the data, both the descriptor and the actual segment files must exist.


Cleaning up Data Sets

A good plan for managing data sets is to identify the Data Sets that are no longer required, and to use the Data Set Management tool to delete them. If you have the jobs that reference the data sets, you can open each of the data set descriptor files using the Data Set Management tool and then view and delete the data set. If you do not have the jobs, another possible method is to look in the resource disk locations for segment files with very old modification dates. Once you have identified the segment files, you can determine what the data set descriptor file name was.

/orch/s0/mydata.ds.user1.host1.0000.0000.0000.1fa98.b61345a4.0000.88dc5aef

In this example segment file shown above, the highlighted "mydata.ds" is the file name of the data set descriptor. You can then locate this file in your computer with the find command.

find /my_projects/datasets/ -name "mydata.ds" -print

Once you have located the descriptor file, you can then use the Data Set Management tool to view and delete the data set. If someone has already deleted the descriptor file, then the segments have been orphaned. There is no utility or function to recreate the descriptor file. In this situation, you can safely delete all the segment files named with the "mydata.ds" in the file name.


Cleaning up Data Sets from the command line

It is also possible to use the orchadmin executable program to delete data sets. This program is located in $APT_ORCHHOME/bin.

APT_CONFIG_FILE=/opt/IBM/Ascential/DataStage/Configurations/config-2x2.apt
APT_ORCHHOME=/opt/IBM/Ascential/DataStage/PXEngine/bin
cd `cat /.dshome`
. ./dsenv
LD_LIBRARY_PATH=$APT_ORCHHOME/lib:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH
APT_CONFIG_FILE=< /opt/IBM/Ascential/DataStage/Configurations/config-2x2.apt>; export APT_CONFIG_FILE
APT_ORCHHOME=$DSHOME/../PXEngine; export APT_ORCHHOME
PATH=$APT_ORCHHOME/bin:$PATH; export PATH
orchadmin describe -c -p -f -s -v -e /dsd/od251dev/data/incoming/9997/20120414/9997_MRKT_FCTR_VOL.ds
orchadmin delete <full path to descriptor file><datasetname.ds>
Link
http://www.isecor.com/kb/documentation/orchadmin.html

Clear out RT_LOG
                The RT_LOG* files are cleared when you clear the log of a job from the Director